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ABSTRACT 

The position papers presented at a seminar held in 
1969 at Leeds University are the basis of this report. The papers 
cover the topics of computers in education, learning systems, 
planning and management, hardware and engineering, development 
processes in computer assisted instruction (CAI) , and economic 
problems. The 46 participants at the seminar were from the United 
Kingdom, Canada, Belgium, Holland, France, and the United States. A 
summary of the discussion of each topic is presented. Some of the 
papers have lengthy reference sections. The discussion of a proposal 
by the National Council for Educational Technology (NCET) for a 
national research and development effort is also recorded. (JY) 
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EDITORIAL NOTE 



This document reports the proceedings of a Seminar on Computer-based Learning Systems 
held at Bodington Hall, Leeds University, 8-1 2th September, 1969. 

The 46 participants from the United Kingdom, Canada, Belgium, Holland, France and the 
United States held 7 sessions in which precirculated position papers were discussed. For 
the first and last sessions the papers under consideration were three documents produced 
by the National Council for Educational Technology. These are not reproduced here but 
are available separately from NCET (for details see page 7). 

The discussions were wide-ranging and informal. It was impossible to record all the 
contributions and exchanges in detail. A summary of the discussion in each session was 
prepared by rapporteurs in which the essential content and some of the original flavour is 

preserved. 

The Seminar was arranged by John Duke, Assistant Director of the National Council for 
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WHY COMPUTERS INEDUCATION? 



J.F, Duke, 

National Council for Educational Technology 



INTRODUCTION 

Several years ago there was a short round table conference in 
London sponsored by ONR on CAI, in which a number of 
experts from the US came over to share their experiences. 
That meeting stimulated people in this country to look 
seriously at the computer as a component of the education 
system. Since then much has been thought and talked about 
in Britain of using the computer in the learning situation, but 
only limited experimentation has so far got under way. 
Meanwhile developments in the USA have been almost 
explosively far-reaching, with over 100 identifiable centres 
now investigating aspects of this field. We in Britain have of 
course tried to keep up with the burgeoning literature 
(although this is often more speculatively descriptive than 
factually informative). We have had the benefit of 
occasional lectures from distinguished practitioners and there 
have been sessions at conferences (such as those organized by 
APLET and IFIP) devoted to CAI and some of us have been 
fortunate enough to see things at first hand in the States. 

It appears to me that the time is particularly ripe to take 
deeper stock, to pool ideas and experiences and to indulge in 
that critical reflection that only a few days cut off from all 
other earthly cares can nurture. When persuading the 
National Council for Educational Technology to undertake 
sponsorship of this meeting I was most fortunate in being 
able to call on the generous support of the US Office of 
Naval Research, whose additional aid is most appreciated. 

This Seminar comes at what may be a critical juncture in the 
British attitude to investigating computer based learning 
(of which more later), but I think I also perceive a wind of 
change blowing through the corridors of the US funding 
agencies. I would like to thank our American guests for 
responding so whole-heartedly to our request to pick their 
brains, and hope that by the end of the week they may not 
feel the balance of payments has been all one way. Further, 
through the Centre for Educational Research and Innovation 
of the OECD, international relationships in this field are 
being explored. It is thus with much pleasure that, to 
balance as it were the transatlantic contingent, we have as 
participants in this Seminar workers from continental 
Europe. This Seminar will I hope be remembered as a key 
gathering this side of the Atlantic and be a precursor of many 
future meetings. 

TheNCET is a largely advisory and exploratory body, set up 
by government, but independent of it, to advance the 
application of educational technology in this country. It is 
not a grant-giving organization and has to exercise its 
persuasive powers to exhort action in others. Right from its 
beginnings two years ago the Council recognized it should 
take a positive role in respect of the computer’s role in 



education. As a result it set up a series of studies which 
produced documents containing a preliminary state-of-the- 
art survey 1 ; a more thorough investigation of the 
feasibility of R & D work in this field and of the criteria that 
should be followed 2 ; and most recently a policy statement, 
addressed to government outlining a recommended plan of 
action 3 . 

I felt that these documents could provide a useful framework 
for this conference. They represent a philosophy of approach 
which I hope will be discussed, although I hope we will not 
get bogged down on too detailed a debate. They highlight a 
number of the points at issue and support directions in which 
we might begin to seek answers. They relate to a peculiarly 
British situation, but this may have parallels elsewhere. 

In this country educational research is something of a poor 
relation. The total amount spent on research and 
development is only about 0.1% of the total annual 
educational budget. Not only are we up against a traditional 
lack of money, there are also shortages of institutions with 
adequate staff and resources to undertake major R & D 
projects. The central Department of Education and Science 
undertakes virtually no innovative research itself and 
directly commissions little more. It affects to stand back 
from influencing both curriculum and method, concerning 
itself with general policies and plans, but often does not 
make the wherewithal available to carry these out. At the 
user end authority is fragmented into units too small to 
sponsor useful developments. The Local Authorities do 
contribute towards the NFER, but its work is largely post 
hoc evaluation. They do now support the Schools Council, 
whose committees are responsible for new curriculum design 
and the need for which arose out of the pioneering work of 
the Nuffield Foundation. The Schools Council only covers 
the primary and secondary sectors — it has no concern with 
the further education field. At universities the effect of their 
educational research on teaching has been infinitesimal. The 
UGC does not normally provide resources for specific 
developments, although it did make available earmarked 
funds for establishing certain audio-visual centres. The SSRC 
can grant-aid specific projects within its small budget, but it 
is more inclined to favour basic research rather than applied 
development. 

In the field we are concerned with, there is no suitable 
mechanism to set work in train such as we see in America, 
Sweden and France. There is also lack of facilities. 

Computers are still in short supply in this country. 
Computing power is inadequate for research into the physical 
sciences (which have traditionally had the lion’s share) and 
there is no spare capacity for educational purposes. 
Educationists are generally ignorant of the capabilities of 
computers and appear happy to remain so. It is against this 
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background that the NCET proposals must be viewed, and 
any alternative suggested. Firstly, government has to be 
persuaded that it matters; secondly the ends have to be 
decided (are we seeking knowledge for its own sake or are 
we seriously attempting to assist in relieving some of our 
pressing educational crises); and thirdly the means for 
achieving these ends have to be set up. 

This brings me to the objectives I see for this conference. I 
would like us first to consider what it is we want of 
computers in education, and how do we best set about 
achieving this. What is the justification for exploring this 
field, and on what criteria can success be assessed? What 
appear to be the major problem areas and where should the 
balance of effort lie? This will lead us on in subsequent 
sessions to consider major sections of activity and the best 
lines of advance in each. These I have suggested may be 
arbitrarily divided into hardware and engineering problems — 
the question of making available a machine system to fulfil 
educational tasks, the software problems — concerning the 
development of appropriate learning materials, and the 
conception and implementation of learning systems through 
which hardware and software combine with teachers and 
students to achieve a viable whole. 

Towards the end of the week we will look I hope at. the 
economic implications of the problems and solutions we have 
been posing — who pays and how. And at the question of 
organizing all the effort that is required, is a disciplined 
approach possible? None of these topics is exclusive in 
itself and all have implications for each other. I hope we will 
be considering these interdependencies for in the final 
session I would like to piece together the themes we have 
been following, to reappraise our problems and priorities and 



look to the opportunities for collaborative action — in this 
field we cannot all afford to re-invent the wheel. 

As to the pattern of the sessions, you will all have pre-printed 
position papers. To those who I have cajoled, flattered, 
bullied or downright insulted into preparing them could I 
publicly display my sincere thanks and appreciation. These 
will be briefly introduced by their respective authors; then 
equally briefly commented upon by the session rapporteur 
who will, I hope, point up areas for discussion into which we 
can all pitch. 

Before I close I would like to record our thanks to our hosts, 
Leeds University. I am looking forward greatly to this 
conference, which I hope will be both critical and construc- 
tive, open-handed and open-minded, and not I hope too 
solemn. I would like to think that by the end of the week we 
have made some real progress to seeing our way through to 
developing computer based learning to a point where it may 
have a truly significant impact on our educational systems of 
the future. 



1 ‘Computers for Education’, NCET Working Paper No 1 

Councils and Education Press, London, 1969 

2 ‘Computer Based Learning Systems; a programme for 

research and development’ 

National Council for Educational Technology, 
London, 1969 

3 ‘Computer Based Learning; a programme for action’ 

National Council for Educational Technology, 
London, 1969 



DISCUSSION 



Launching the proceedings Mr. K. Hill, who with Mr. I 
McMullen led the team that carried out the analytical study 
on which the National Counci 1 based its proposals, spoke of 
the rationale behind the approach taken in their report 
‘Computer Based Learning Systems — A programme for 
research and development’ (NCET, 1969). This was to view 
computers in education against the whole achievement in 
education and to consider their potential use by a 
subjective projection of what might be developed both 
technologically and educationally. Secondly, they examined 
the consequences of problems and decisions that would arise 
from the development, both in the UK and abroad, of the 
use of computers in education, and derived the nature and 
hence the likely cost of development both to meet particular 
educational aims and for a minimum defensive strategy. 

Very limited use of the computer was foreseen at present in 
education up to the age of 1 6 years except possibly in the 
computer managed instruction mode. Post-16, and in adult 
education and training, its use was foreseen in courses with 
logically structured content and where its simulation and 
problem solving facilities could be valuable. However careful 
curriculum analysis was a first essential to determine the 
place and kind of contribution the computer could make and 
it was an essential pre-requisite to view the computer as a 
component of a learning system. 

As a result of their analysis of the relevant social and 
economic criteria Mr. Hill and his co-workers had put forward 
an outline R & D programme in two main parts: a short term 
programme aimed at gaining a perspective about the applica- 
tion of computers to education, and incidentally helping to 
meet immediate educational needs, and a long term 
programme aimed at exploiting computers and associated 
technologies to meet the educational needs of the nation and 
prepare a defence against any threat to educational values by 
commercial or foreign dominance in this field. The 
recommended short term programme should concentrate on 



tile development of CAI/CMI courses, mainly in service 
subjects, in universities utilizing in the main existing 
computer installations, and that this programme should be 
centrally organized and coordinated. 

In their subsequent submission to Government, the National 
Council for Educational Technology, while agreeing generally 
with tli . underlying philosophy of the study team’s report 
and their analysis of the situation, had taken the view that 
the R & D programme should be more broadly based, and 
this was regarded by many as dissipating the available 
resources too thinly. This point was taken up again in tjie 
concluding session. Meanwhile there were a number of 
criticisms of the feasibility study. In particular it had taken 
no account of experimental psychology. In accepting the 
psychologists’ role in educational research Mr. Hill did not 
consider that a psychologist on the team would have added 
very much. In his view the rate of advance of psychological 
insight -into the learning process was insufficient to allow of 
an end result in a finite time, and would probably mean that 
psychological investigations would in the main have to be 
outside the scope of the short, term ‘perspective’ programme. 

On the question of using existing computing resources there 
was much argument in favour of experimenting with smaller, 
modern machines rather than with the obsolescent KDF9’s, 
but here again there was insufficient evidence to specify 
precisely what was required of the computing system and any 
any formulation decided on today was likely to be outdated 
by the time implementation was a practicality. Doubts were 
also expressed as to whether the sums proposed which should 
be spent on R & D were sufficient, bearing in mind the 
considerable investment the USA was making in this field. 
The answer here was that there was at the moment an 
inadequate basis for investment. Although there were 
considerable claims for the computer by experts, what insight 
did they have and was this commendable enough to persuade 
the policy makers? The object of the ‘perspective’ programme 
was to provide a sound basis on which future, and more 
substantial, investment might be made. 
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TECHNICAL CONSIDERATIONS IN THE DESIGN 
OF A CAI COMPUTER SYSTEM 



E.N. Adams 
Research Division, IBM 
Yorktown Heights, New York 



I. GENERAL INTRODUCTION: CONSTRAINTS 

This paper is a discussion of technical characteristics of CAI 
hardware and programming systems in relation to present or 
foreseeable future developments of CAI. The aim is to 
enumerate and put in perspective major considerations that 
might influence the inclusion of given capabilities in the 
specifications of a CAI system. No attempt will be made to 
deduce a relatively unique set of system requirements as, for 
example, through analysis of the cost-benefit tradeoffs 
between what is desirable in terms of psychological, 
educational, and social considerations and what is, or can be, 
available in terms of state of the art, technological develop- 
ment, and practicalities of support of operational instruction; 
indeed in the present state of development of CAI the future 
costs and benefits are still to a considerable extent hypo- 
thetical. 

The starting point for the design of a CAI system is the set of 
external constraints imposed by the larger environment in 
which the CAI system is to operate. Such constraints derive 
from the purposes of the institutional sponsor of the project 
that will operate the CAI system and the amounts and types 
of resources available. A primary constraint is likely to be 
the computer system to be used, its associated equipment, its 
satellites and terminals, and its operating system environment. 
A second set of constraints is the aggregate of courses and 
services that must be furnished, the operational loads of 
various kinds, and the schedules and the physical locations of 
the users and terminals. Finally, there are such constraints as 
the kinds of staff available for system maintenance, develop- 
ment, and operations, and the administrative organization 
and division of labor within the system. 

Trends in Equipment 

Disregarding what might be most desirable from the view- 
point of CAI methodology, the systems capabilities being 
developed in connection with general time-sharing and 
remote entry techniques seem likely to provide the frame of 
reference within which most CAI systems will be developed 
in the near future. Within that framework we can project the 
characteristics of equipment to become available in the next 
years. For convenience we will review these under the 
headings of terminals, CPU/OS environment, and application 
programs. 

Terminals 

A number of types of terminals have been used successfully 
with one or another kind of CAI. Each has generic physical 
and logical characteristics that determine what kinds of 
instructional techniques and/or systems configurations it is 
suitable for. We will enumerate a representative set of such 
terminals and review them in relation to the following: 
limitations of input messages 



limitations of output messages 

remotability 

special advantages or disadvantages 

1. Teletype. Remotable; uses low cost lines; both input 
and output messages are alphameric, prints in a single case 
font. 

2. Telephone with touch-tone attachment plus voice 
answerback. Remotable; input messages are keyed 
alphameric; output messages are voice messages transmitted 
through an ordinary telephone circuit; advantage is a very 
low cost terminal with a developed distribution system in 
place; disadvantages are lack of hard copy records; costly 
central system features, limited flexibility of output 
messages. 

3. Two-case typewriter with changeable print element. 
Similar to teletype, but is more flexible in a multiple appli- 
cation CAI system because of capability to quickly change 
symbol sets. 

4. CRT-keyboard. Sometimes called a ‘soft typewriter,’ 
both remotable and non-remotable form; input and output 
messages normally in single font of alphameric. Advantages 
over printers: lends itself to font expansion at extra cost, in 
which case simple graphics may be displayed; erasure is 
possible, a light pointer attachment may be used as an 
auxiliary input, display format is much more flexible; 
disadvantages are a relatively small field of output that may 
be viewed at one time, look-back is usually costly, no 
permanent output copy. 

5. CRT with keyboard, light pen or RAND tablet, and 
vector graphics. Not remotable, except at great cost of speed 
or bandwidth; all capabilities of soft typewriter; in addition 
input messages can include analog data in the form of light 
pen strokes, output messages can include depiction of 
relatively complex curves, graphs, display of functional 
relationships; disadvantages, high terminal and system costs. 

6. Electronic blackboard with keyboard, light pen, and 
stored images superposed on CRT display. Not remotable 
except at great cost of speed or bandwidth; output display is 
a composite of stored visuals and digitally generated informa- 
tion; input can include touch pointer that refers to this 
composite; has special convenience advantages for program- 
ming of simple learning tasks; also the composite image 
permits unique learning tasks; disadvantages, high terminal 
plus system hardware cost. 

7. Plasma tube display with stored visuals. Functionally 
similar to the electronic blackboard, but at a lower cost; as a 
storage device presents different addressing, reading problems 
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from the CRT. 

8. Random access, visual projector. Sometimes a 
supplemental display attached to a printer or CRT terminal; 
with touch sense capability it can stand alone; remotable; 
advantages are low signaling rate; disadvantages are limited 
message set. 

9. Random access audio output. Supplementary to 
printer or CRT display; advantage; low signaling rate; 
disadvantage; randomly selected audio messages are relatively 
few in number and/or accessed rather slowly. 

10. Other analog attachments can be used as terminals or 
terminal attachments. Advantages are special functions that 
can be realized; disadvantage is relative inflexibility ; most 
useful where the specialized equipment can be amortized 
over many students. 

Certain engineering considerations are fairly fundamental in 
connection with communication between CPU and terminal. 

Data Rates 

Maximum input data rates for keyed devices are in the range 
10-30 bits per second; thus in principle many keyboards 
could be multiplexed to use a single voice grade telephone 
channel as a link back to central. 

The output data rate required to service a terminal depends 
both on the average size of the messages sent to the terminal 
and the form in which the message is transmitted. In existing 
CAI work output messages might range from less than 100 
to 5000 bits in length if transmitted in symbolic form, with 
1000 being a typical value. If transmitted in analog form, 
these same messages would consist of 1000 to 50,000 bits 
and thus are a factor ten larger than when in alphameric form. 

The actual loads on the communication channel depend on 
the frequency with which such messages must be sent, which 
depend, in turn, on the instructional design of the CAI 
program. I do not have good statistical information with 
which to characterize message lengths and frequencies in CAI 
programs, so I can only offer my personal impressions. 
Student response times range from a few seconds to a minute, 
and bear some crude relation to the complexity of the 
student tasks; as a rule of thumb one can estimate the total 
student response time as several seconds for each character 
keyed for very short responses to about one second for each 
character keyed for long responses. In a similar spirit we may 
adopt the rule of thumb that the number of characters 
displayed in an output message ranges from one to five times 
the length of the student response. (There is, of course, no 
effective upper limit on the length of output messages if one 
uses the computer to print out lectures or multiple choice 
tests.) Combining these rough rules we can estimate that the 
mean two-way load on a data link is a few dozen bits per 
second when messages are transmitted as alphameric 
symbolism; and about a factor ten larger for those parts of 
the message transmitted as video bits. 

Remoting 

When terminals must be located far from the CPU, the need 
to economize on the communications facilities becomes a 



factor in computer system design. Communication is most 
often by means of the commercial telephone system voice 
channels, often on a single terminal per channel basis. Since, 
however, a voice grade telephone channel will transmit up to 
9600 bits per second, in principle it could accommodate a 
load up to 100 terminals. Thus, where communications 
distances are large enough, it may become worthwhile to 
provide multiplexing equipment so that a cluster of 
terminals can be serviced via a single voice grade channel. 

Remoting of typewriter type terminals is straightforward. 
Remoting of CRT type terminals involves a variety of 
considerations, depending on how the symbols or other 
figures displayed are generated and depending also on how 
the CRT display is regenerated from scan to scan. 

In the simplest CRT systems, which use a raster scan, there 
are actually three processes to be considered: the generation 
of the video representation of a symbol from its symbolic 
representation, merging of the individual symbol video into 
the video for an entire raster, and the process of retaining the 
information to recreate a display from one CRT scan to tne 
next. The video for an individual symbol may be generated 
by a table look-up procedure either in the CPU or in any of 
several types of read only memory located at or near the 
terminal. The merging of the individual symbol video into 
the raster video is an electronic procedure done in the 
terminal. The information to ‘refresh’ the CRT on a new scan 
may be retained in symbolic form, in which case new video 
must be generated for each scan, or it may be retained as 
video in some kind of synchronized memory, e.g., a delay line 
or rotating magnetic store. 

Remote location of the terminal interacts unfavorably with 
those system organizations in which the CPU is involved in 
creating the video, to the extent that it requires a high 
bandwidth connection between CPU and terminals. Assuming 
high bandwidth is not available, CPU generation is not 
practicable unless a slow average rate of video generation is 
acceptable, and the video for ‘refresh’ is stored at the 
terminal. 

Special Fonts 

The need for special or extended fonts is very common in 
CAI applications. Language, mathematics, and the sciences, 
all generate needs for special alphabets or special symbols. 
With printer type output devices special fonts can be 
achieved by use of special printing elements; extended font 
size, however, is essentially impossible. 

With CRT type output devices several possible approaches of 
achieving an enlarged font may be considered, depending on 
the engineering means by which the CRT displays are 
generated and maintained. If the means of providing the 
enlarged font is very costly, it may need to be shared among 
a number of terminals; thus when large font size is achieved 
by a hardware approach, one must then multiplex the 
expensive equipment that generates the video symbols; when 
it is achieved by a CPU software approach, one must arrange 
to have a single high bandwidth channel service many 
terminals, and possibly also arrange for a single video 
regenerator to serve many terminals; when it is achieved by a 
satellite CPU approach, the small satellite CPU is shared 
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among many terminals. When the physical distance between 
the computer and the terminal is large (100 miles, say) so 
that it is too costly to provide video bandwidth channels to 
individual terminals, the high bandwidth channel must be 
multiplexed; in such cases the technical approach to 
generation of displays and regeneration of displays clearly 
interacts with the multiplexing extending scheme. 

Hard Copy 

In many situations it is important to have a simple means of 
preserving protocols of student performance at the terminals. 
With printer output technology this is reasonably straight- 
forwr- d to do. With CRT output technology there is no 
solution that appears to be practicable and fully satisfactory: 
system log records do not provide a convenient means of 
reconstructing what was displayed; photography and 
facsimile are neither convenient nor economically feasible. 

CPU/OS Environment 

Perhaps the single largest constraint for the typical CAI user 
is the computer system. We expect that the typical system of 
the future will be a time-shared multi-terminal system. We 
Envision a system having a fast memory of at least moderate 
size (> 100K bytes), a rather large (hundreds of millions of 
bytes) amount of disk'type fast access storage, multiplexing 
and transmission control units as appropriate for its terminals, 
and various other system components as necessary to operate 
such a system. 

As important as the hardware capabilities of the computer 
system is the Operating System (OS) environment that it 
provides. The OS provides the programming system that 
manages the machine. It schedules all CPU operation. It 
allocates all system resources. It provides the interface 
through which programs communicate with terminals and 
satellites and also with the peripheral storage. Thus the OS 
provides the communication interface between a program 
and all of its sources of data, and determines when and 
whether the program will be executed. 

Important constraints are implicit in the languages and 
procedures provided by the OS for requesting the execution 
of jobs. Thus the conventions of requesting a job are based 
on some view of what types of request are routine or typical, 
what types are non-routine or unusual, what types need not 
be or cannot be processed at all. Priority procedures in a 
system are based on a view of how a system is normally 
operated, what kinds of users there are, what are the responsi- 
bilities of various users, etc. The designer’s conception of the 
user that is implicit in the OS, thus has a strong structuring 
effect on the user. 

The OS environment is determined not only by the charac- 
teristics of the supervisor but also by those of what we may 
call the subsystems under the supervisor. For this purpose a 
subsystem is a set of routines that control entry and re-entry 
to a class of programs. Since the rules for controlling entry to 
programs are related to user prerogatives, each subsystem 
may be thought of as associated to, or as defining, a 
particular class of user. Specific to any subsystem are 
routines to validate sign-on requests and secure necessary 
files, effect proper updating of records at sign-off, and error- 
handling routines. 



Imp? ait. at subsystems in a time-shared CAI system include 
subsystems for use by system programmers, operators, and 
administrators, a subsystem for application programmers, a 
subsystem for execution of general applications programs, a 
special instructional monitor or interactive CAI subsystem. 

Below the subsystem level there are programs. A program is a 
set of routines that performs a well defined job. Just as 
subsystems correspond to users, programs correspond to user 
services. Among the programs necessary or desirable to have 
would be, a meta-compiler, a macro-compiler, a number of 
compilers associated to the various processors available to 
run under the system, a file manipulation program, a 
statistical analysis program. Processors might include one or 
more languages like FORTRAN or PL/ 1 , a conversational 
interpretive language like BASIC or APL, a statistical 
processor, a list processor like LISP or SLIP, a CAI author 
language, special course compilers, etc. 

Courses 

A last major set of constraints on the design of one’s system 
comes from the courses in which CAI is to be used. While in 
principle the decision to use CAI in a given course might be 
thought of as a purely pedagogical one, in practice the 
decision is arrived at through a process involving many other 
kinds of considerations, although presumably CAI would be 
decided on only where it is pedagogically feasible. 

Broadly speaking the courses to be taught will be constrained 
by some set of given instructional objectives, to some part 
of which CAI methodology is applicable. The overall instruc- 
tional plan will ordinarily involve the use of resources in 
several media of which CAI is only one. The plan of how the 
various learning objectives are to be allocated among the 
urious media and how the media are to be articulated will 
probably be designed to optimize the use of media resources 
while meeting pedagogical objectives, rather than merely to 
optimize pedagogy. 

In courses that use a variety of media, the media units will 
ordinarily have to be packaged in modules of somewhat 
arbitrary size for logistical reasons: if a learner is to be 
scheduled on the machine, he must be assigned a quantum of 
work large enough to avoid excessive losses of student and 
machine efficiency through various overheads such as student 
travel time, proctor operations, tardiness, machine time 
unused because students finish e^rly, etc. This modulariza* 
tion, too, is an external constraint in that it is not generally 
derivable from pedagogical considerations alone. 

II. INSTRUCTIONAL CONSIDERATIONS 

The kind of external constraints discussed above define the 
frame of reference within which a CAI system must be 
designed; what to do within that frame of reference should 
be decided on the basis of psychological and pedagogical 
consideration. Fundamental among these considerations is 
the nature of the learning process. 

The concept of learning as a process of operant conditioning 
is especially useful as an approach to instruction in that it 
suggests principles of design for learning programs that are 
largely independent of the underlying physiological 
mechanism of learning. The basic model of operant con- 
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ditioning is: 

1. The learner engages in a range of behavior. 

2. The environment responds; the topography of response 
corresponds in some sense to the topography of the learner’s 
behavior. 

3. Perceiving the topography of environmental response, 
the learner is able to assign different values to different 
variants of his behavior. 

4. As the cycle of behavior and response is iterated, 
behaviors of high value occur with increased frequency. 

The differential response of the environment to different 
behaviors plays a central role in the shaping of operant 
behavior. The learner learns to differentiate between two 
behaviors only if some contingency in the environment 
differentiates between them. One may say that what a 
learner learns about an environment is precisely its system of 
behavioral contingencies. 

Consonant with this view of learning, instructional design can 
be approached as the design of a special learning environment 
in which the contingencies of reinforcement are designed to 
select and enhance the behaviors that are the goals of instruc- 
tion. Thus a learning environment is to be designed about the 
particular class of behaviors it is intended to shape. 

The basic building block of a learning environment is the 
learning task. A learning task is a stereotyped synthetic 
activity with arbitrary conventions, rules, and objectives. Its 
principle of design is to create a situation that demands the 
intensive exercise of a particular behavior on which the con- 
tingencies of reinforcement directly depend. The learning 
task is in effect a very specialized communications medium. 

Its rationale, rules, etc., narrowly limit what is communicated 
about and what kinds of messages are exchanged, and create 
a set in which relatively simple stimuli suffice to communi- 
cate a complex topography of response. 

The nature of a learning activity may be more or less artificial, 
depending on the extent to which the behavior being 
developed is a subskill of the final behavior or a rough 
approximation to it. In any case the learning task itself has 
content and involves behaviors that are irrelevant to the final 
instructional goals, but are necessary for purposes of doing 
the task; examples of such secondary skills are the use of a 
keyboard, the format rules of messages, the procedures used 
in a test administration. Irrelevant as these secondary skills 
may be they are part of the instructional content, in effect 
an overhead associated with the use of the learning task. A 
great amount of student difficulty may be caused by the 
failure to give enough attention to the design and communi- 
cation of this kind of content; conversely a major advantage 
to the iterative use of learning tasks of stereotyped form is 
to minimize this type of overhead and to achieve good 
secondary skills on the part of the learner. 

Since the effectiveness of a learning task depends on the 
efficiency of communication it achieves, the focus of learning 
should be narrow; the learner should not be required to give 
significant attention or effort to behaviors irrelevant to the 
instructional goals; and the quantum of learning should be 



kept small enough so that environmental stimuli can be 
properly and confidently evaluated by the learner. At the 
same time the environmental stimuli should be rich enough 
to convey the appropriate topography. 

The keys to the effectiveness of a learning task are three. 

1. Stimuli to evoke and reinforce the behavior in question. 

2. Means to sense and evaluate the learner’s behavior. 

3. Algorithms of control to select appropriate stimuli. 

A learning program leading to a relatively complex final 
behavior will commonly consist of the iteration of a single 
learning task with a continual change of content. Several 
levels of valuation are likely to be needed in such a case: 
message processing, task valuation or scoring, valuation of 
learner state, valuation of learner performance over various 
periods of time. 

Conventional instructional systems depend on humans to 
carry out the crucial functions of response valuation and/or 
stimulus selection. Where learning tasks can be devised that 
use a computer terminal, interactive CAI could in principle 
take over the valuation and/or selection functions if a task 
valuation algorithm and an algorithm of control can be 
made. Numerical valuation systems, i.e., scoring systems, 
are of special value and importance because they lend them - 
selves to mathematical modeling. 

It is useful to think of a CAI program as having three separate 
aspects, which may or may not be clearly articulated in the 
code. We will refer to these aspects as the content , the 
communications media (mediation for short), and the control 
aspects. The content aspects concern the scope of knowledge, 
behavioral objectives, the sequence of learning tasks, etc. 
Mediation aspects concern the articulation of media, 
mechanics of learning tasks, message processing, stimulus 
generation, etc. Control aspects concern scoring, indices for 
valuation of the state of learning, criteria of allocating effort, 
etc. 

It is sometimes possible to structure a CAI program so that 
material relating to each of these aspects is localized in its 
own separate section of the code. Such a program structure, 
when it can be achieved, has a number of desirable features, 
including ease of preparation and revision, and ease of dis- 
semination. A further step in the same direction is the separa- 
tion of the control programs associated to the learning task 
into two parts, one which is concerned with message pro- 
cessing, task scoring, and stimulus selection or generation, 
and a second associated to higher level control, which is con- 
cerned with criteria of mastery, indices of mean performance, 
sequencing of learning tasks, modules, and lessons. 

Where valuation must be applied to a complex topography of 
behavior, task scoring can be multi-dimensional. Corres- 
pondingly where it is necessary to construct stimuli that dis- 
play a complex topography of response, a multiplicity of 
scores can be combined, as in a multifactor regression 
formula, to specify values of a number of control variables. 
For operational reasons it is convenient to cast such multi- 
factor valuation and control algorithms into forms in which 
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adjustable parameters provide means to control and alter the 
algorithms; a simple example is a scoring formula in which 
various factors or criteria are combined in a weighted 
average, the weights being adjustable by the instructional 
supervisor. 

The extensive use of numerical scoring and numerical indices 
of control facilitates the use of program structures in which 
response processing routines are clearly separated from the 
learning task control routines. Programs so structured are 
relatively simple to prepare. They also provide operational 
flexibility, since the instructor can readily change the con- 
tent of a parameterized exercise without changing the 
algorithms of scoring and control, or conversely these 
algorithms, but not the content. 

A parameterized control program of fixed form is a means of 
adding quantitative dimensions to the concept of instruc- 
tional method; different materials can be used with the same 
control program, different control programs with the same 
materials, relative effectiveness can be studied as function of 
control parameter settings, etc. Parameters of a control pro- 
gram can provide convenient dimensions along which to 
optimize the effectiveness of learning tasks; they can also 
provide an approach to self-adaptive optimization of learning 
tasks. 

Thus far the discussion has been independent of the nature 
of what is to be taught. However, it is clear that behavioral 
goals are of considerable variety, and a learning task must be 
accommodated to the nature of what is to be learned. One 
taxonomy (Bloom’s) categorizes behavioral objectives as 
effective, cognitive, or psychomotor and, within the cognitive 
domain, as ranging from fact recognition, recall, and know- 
ledge of generalizations to comprehension, application 
analysis, synthesis, and inference. To this wide range of 
types of behavior there must correspond an equally wide 
range of instructional strategies or learning tasks. 

Almost any academic subjects or courses of instruction 
includes among its objectives a considerable range of variety 
of behaviors, so normally requires a composite methodology 
of instruction. In par ticular most academic subjects involve 
a range of cognitive behaviors. Thus it is somewhat frustra- 
ting to be asked, as CAI workers often are, “Can CAI be 
used to teach x?” where x. may be sex education, science, 
art appreciation, “you name it.” Clearly, the question does 
not suggest much understanding of the technical problems 
of instruction. Nevertheless, one cannot just bruskly dismiss 
anyone who asks such a question, especially not when he is 
a Very Important Person; one must somehow give an answer 
if only in the form of a discussion. 

Clearly the first point to make is that the question could 
better address the type of behavior to be learned rather than 
the type of information content the behavior has reference. 
Second, it seems that in most academic courses a number of 
phases are involved — call them presentation, practice, 
recitation, and test — and CAI is not equally applicable to 
all of these, especially when the economics of its use is taken 
into account; thuj the point may be made that it may be 
most favorable to use CAI vis-a-vis other media in the 
practice and recitation phases, whiqh involve the highest 
degree of two way communication, inasmuch as CAI is the 



only completely non-human medium that provides for fast 
flexible feedback. 

Finally, one may make the point that the applicability of 
CAI to various behaviors is limited by the repertoire of 
existing or readily constructable CAJ learning tasks: one 
may enumerate some classes of learning tasks to give an 
impression of what kinds of instructional approaches may 
be possible: 

rote practice or drill 
application practice 
practice with simulators 
presentation and immediate test 
computation 

program compilation and execution 
game playing, decision making 
enquiry, exploration, experimentation 
dialog 

test and prescription 
III. PROGRAM FEATURES 

CAI makes use of a number of th. subsystems of the time 
sharing system. The features needed in these subsystems are 
for the most part similar to those needed for general TS use 
and may not require comment. However, some of the features 
at the program level deserve discussion. 

Author Language 

An important system feature is the author language in which 
CAI programs are to be prepared. In principle the author 
language might be any of a number of high level languages. 
However, in fact the need to execute CAI programs under a 
special student mode subsystem (instructional monitor) and 
to execute them with high efficiency, implies that it is 
advantageous to use a special author language for coding 
CAI programs. The view taken here is that convenience 
features of the author language itself are secondary to its 
functional capabilities. Key functional capabilities include: 
Variables 

arithmetic, including floating point 
logical 

alphameric strings 

data structures having programmer specified format 
lists 

The number of variables needed for a single student’s use in 
a single course is at least 10 sentence-length ('v 100 ch) string 
variables, several dozen numerical variables, and at least 100 
logical variables in the kinds of programs we have worked 
with. We have sometimes wanted more. 

Operators 

arithmetic 

logical 

boolean 

character 

pattern operations on strings/lists (see below) 
special operations on data structures (see below) 

Control Statements 
conditional: IF 
iteration: DO 
transfers: GO TO 
subroutine: PROCEDURE 



storage allocation: DECLARE 
restart point 
time dependent transfer 
log entry 

Macro processor within the author language 
Library routines written in other languages 
which can be used in author language 

A CAI language must provide a variety of message processing 
functions. Messages may consist of natural language strings, 
data structures, and numerical or logical arrays alone or in 
combination. 

Important processing operations on natural language strings 
include: 

Definition or detection of substrings 
specific substrings 
fields, e.g., first numerical field 
defined by lists or sets 
defined by context 

Transformational processing 

edit, e.g., replace, delete, affix 
re-order substrings 
insert substrings 

Tests on numbers or strings (vs. reference string) 
character match 
numerical equality 
numerical range match 
degree of match 

pattern tests (combinations of order and match) 
general user defined test 

Processing of data structures is useful in connection with 
coding for learning tasks of stereotyped form, for which the 
format does not change over a number of learning tasks. The 
existence of a macro processor provides for a part of the 
desired capability. However, it is desirable to have flexible 
means to form such structures during program execution, 
and to provide in general a compactness of code that will 
facilitate rapid execution. 

The instruction to make a log entry is one with peculiar 
significance to educational applications, especially where 
recording of program status and of the occurrence of 
particular transactions are needed as a basis for research 
or developmental analysis of the process. The basic operation 
required is “write into auxiliary storage a record of specifi- 
able form” when a specified condition occurs in the execu- 
tion of the program. Logging capability should interact with 
the capability to call for certain routine statistical analysis 
in connection with program execution, since much subse- 
quent searching, sorting, and other format processing can 
be avoided if the records can be put into optimal form at 
the time they are logged. 

Logical processing to detect patterns in stored data may 
become important as the techniques of utilizing more com- 
plicated algorithms of control develop. The capability to 
manipulate and compare subpatterns within boolean vectors 
or arrays is very desirable; the capability needed is analogous 
to that needed for dealing with numerical variables in indices 
of performance and numerical algorithms of control. The 



capability to assemble rhetorically acceptable natural 
language statements incorporating various informational 
elements under a number of constraining conditions is very 
desirable for purposes of using natural language messages as 
stimuli. 

Program Preparation 

Large amounts of stored program are required to support the 
CAI portions of a course, typically of the order of hundreds 
to thousands of bytes of machine code per minute of instruc- 
tion. The process of creating programs of this magnitude that 
will be satisfactory in regard both to content and pedagogy 
can easily be a very costly one. 

One line of thought is to accept the intrinsic costliness of 
CAI programs and to seek means to amortise the large 
preparation costs over large enough number of students. 

Such an approach implies a high degree of standardization 
of both systems and courses, a degree which could inhibit 
the development of CAI both quantitatively and qualita- 
tively and considerably limit its desirability and utility. 

Thus the alternative of lowering the cost of the preparation 
of CAI programs is a very important one. 

Several of the early CAI. workers have tried to simplify 
course preparation by providing a supposedly simple and 
easy-to-use CAI author language. This work has been useful, 
but to the extent that it has conceived the CAI “author” as 
a “subject specialist — turned — programmer,” it has failed 
to develop all of the kinds of programming aids that are 
needed. 

To elaborate this point: the process of producing completed 
CAI programs involves the efforts of several different kinds 
of professional/technical people, viz., content matter 
authorities, learning program designers, computer program- 
mers, computer coders, computer operators, and media 
technicians. Nothing prevents the skills necessary to properly 
perform all of these roles from residing in one creative 
person; however, nothing makes it likely that a high degree 
of all s kill s will in fact be united in one person, or that it is 
practicable to predicate the preparation of CAI programs on 
the availability of adequate numbers of such persons. It 
would obviously be more desirable if program preparation 
could be approached as a team process, utilizing a division 
of labor that permitted various specialists to each efficiently 
exercise his particular skills and do his part of the job with- 
out having to become accomplished in the skills of the other 
team members. 

More important even than the variety and heterogeneity of 
the technical task required for production of a CAI program 
is that these technical tasks will not necessarily or even 
usually be carried out at the same time. Thus the learning 
task designs used in a particular program will presumably 
have been largely worked out before a program could be 
designed around them. Similarly a control program design 
once established as sound may be used in one course after 
another. Thus the production of an actual course should be 
viewed rather as the integration of various technical elements 
than the creation of them. 

The consequence of a proper division of labor is to achieve 



an essential independence of the form and the content of the 
instructional program. When such a separation has been 
realized, the formal aspects of the program are seen as alone 
constituting the method or pedagogy. The development pf 
methods as abstract forms is therefore a logically separate 
activity from the development of courses that use the 
methods; even when the two are carried out at the same 
time, they should be conceived as disjoint activities. 

Certain earlier remarks in this paper about how a CAI pro- 
gram should be structured were arrived at by considering 
what features and procedures lend themselves to the division 
of labor in program preparation. To summarize and elaborate 
these: 

Content should be prepared in a form independent of 
code and convenient from viewpoint of a content 
specialist. 

The essential control program to administer a learning 
task should be localized in a routine; this control 
program should be free of specific content material, 
which should be furnished to it at either compile or 
execute time in the form of a data-structure. 

The answer processing and conversation handling parts 
of the Gontrol program for a learning task should be 
separate from the scoring parts; where appropriate the 
scoring parts should be parameterized and arranged so 
that the parameters may be readily changed without 
any recoding. 

The control programs governing higher level (module/ 
lesson course) scoring and selection of assignments 
should be separate from the learning task: programs 
and from one another; they should themselves be 
essentially content free except for the tables of para- 
meters and other content data needed for execution 
of the course. 

The concept of a CAI program as an appropriate content in 
a stylized form, is consistent with the goal that the various 
members of a curriculum team carry out their individual 
roles with a minimum of mutual interference or hampering 
contingencies. The key to the proper coordination of the 
individual contributions in the production of a CAI program 
is the production director; like the director of a movie, the 
production director is responsible for the overall concept 
of the course and for working out with the other workers 
the broad lines along which technical problems would be 
solved. 

In order to reduce the production costs of such a CM course 
as discussed here a number of programming aids can be 
utilized. The content specialist can use a special text pro- 
cessor that takes manuscript material prepared in a form 
most convenient to him and puts it in a form convenient 
for compiling into a “source” program. The computer 
professional responsible for compiling the program can use 
one or more special “course compilers” which actually put 
the source program together. Each such special course com- 
piler is properly viewed as a special user oriented language; 
these user oriented course compiler languages are the key to 
eventual successful CM production and operation. 



The system operator will have an assembler or compiler for 
producing the program in the form in which it is stored 
when ready for execution. Finally, an author can make use 
of a special software that permits him to make an on-line 
examination of the operation of the completed program, 
to pass control about in the program at will, and to permit 
on-line entry of corrections to the program. 

In some discussion of CAI the source language for coding 
programs, the programming aids, and various services available 
within the computer system as well as procedural and other 
features of the OS environment are discussed under the 
rubric “language.” It is quite important, however, to produce 
programs in such a way as to make the final product useful 
to users of a number of systems having quite different 
features. For that reason one should analyse the production 
process so that only the source language processors are 
needed to make use of the final programs. 

Operational Aids 

A number of different kinds of users make demands on the 
operation of a CAI system. As a minimum these include 
students, proctors, machine operators, and supervising 
teachers. The needs of these users are to be understood in 
terms of their duties, qualifications, and normal working 
procedures. 

Proctors 

Students should be supervised when using the system by 
someone capable of making at least minimal judgments 
about, e.g., the probable source of apparent system mal- 
function as between terminal hardware, system software, 
or course software; and of taking minor corrective actions 
in relation to matters in which fully automatic operation of 
the system is not satisfactory. Proctors should be responsible 
for furnishing extra materials, tape or visual cartridges, etc., 
and of getting various routine reports from the system. 

In the interest of effective proctoring as well as operational 
efficiency, the students should be scheduled into the lab so 
that insofar as possible a number of students are using the 
same course at the same time. 

To keep proctoring costs in control it is desirable that 
proctors be limited to tasks requiring only a low level of 
technical skill and that one proctor be able to supervise a 
number of students. This in turn requires that terminals be 
grouped in clusters, the minimum desirable cluster size being 
the number of terminals that a single proctor can supervise. 

Operators 

In order to maintain a high level of availability for the system 
an operator needs a number of aids. He needs to have a con- 
venient means to quickly determine at any time what system 
services are in use, and who and where the users are. He 
should be made aware of any change in the system, service 
immediately when it occurs. He should have a schedule of 
system services expected to be required, with suitable inform- 
ation about required programs and data to be loaded. 

The operator should have a number of aids for facilitating 
quick and clean restart of a system that has malfunctioned. 



He should be able to obtain summaries of various user 
records and should on properly authorized request be able 
to directly enter needed changes into user records. 

The operator needs a considerable number of utility pro- 
grams for performing such functions as starting, loading 
program; performing various operations on courses such as 
compiling, amending, reassembling, etc.; registering, purging, 
listing records; various control operations to terminals or 
users; etc. The number of such utilities needed depends on 
the system design, but is of the order of four to ten dozen 
in the systems we have worked with. Finally, in the common 
case that the terminals are at a difficult physical location 
from the machine, the operator needs a means to auto- 
matically answer enquiries about the status of the system at 
times when service is interrupted or is not scheduled. 

Supervising Teachers 

Presumably the teacher of a course will not have as much 
close interaction with the system as the proctors and 
students, but will require certain information to be saved 
and summarized for him. The operational needs for sum- 
maries are primarily for student status information, statis- 
tical summaries of use for course administration, and certain 
special information such as “comments” that may require 
short term action. 

Where research or program revision are teacher objectives, 
transaction logs may be needed, generally sorted and cate- 
gorized for the purpose at hand; in addition in such cases 
there will be a need for programs to prepare transaction 



logs for processing by statistical analysis programs. 

Students 

The kinds of student aids needed in a system are a function 
of equipment used, supervision procedures used, and the 
operating procedures of the system. However, in general, 
students need to be able to get certain kinds of information 
at any time; status of the system, status of his terminal (is 
it alive?), explanation of any procedural messages he receives 
(as, for example, when he has violated a format rule), a very 
simple means to obtain a complete account of any proce- 
dures he is expected to use. 

It is desirable that the system have only the simplest pro- 
cedural rules for students and that it be tolerant of student 
message formats. In this regard procedures for erasure and 
cancellation should be simple, and the system should auto- 
matically edit to avoid confusion from entry of redundant 
or ambiguous features of messages (simple example: the 
space-backspace pair within a message). 

Terminal operations should make it clear when a message 
has been accepted, who has control (student or computer), 
(on a CRT) where a message will appear if input is made, 
etc. 

The foregoing pages are an attempt to indicate the kind of 
considerations involved in smooth operations. This discussion 
is necessarily incomplete and fragmentary; good operations 
are achieved as the result of good management plus providing 
a large number of small system features found helpful to 
adapt the system to its users’ needs. 
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INTRODUCTION 

Caveat 

The instructional environment demands of a programming 
language features not easily satisfied by any one implementa- 
tion. Some users will program instructional modules central 
to the system; these require efficient and probably re-entrant 
machine code. On the other hand a student may be asked to 
modify a program or write a new one himself as part of an 
interaction; in this case an interpreter probably serves better 
than a compiler. Furthermore, the executive system may 
itself hinder or preclude desirable elements, such as general 
classes of files. Such a variety of requirements forces a 
general discussion into a framework of general features for a 
hypothetical programming language, comparing existing 
languages against the paradigm. 

Of six languages considered here (Table 1) some are compiler- 
implemented, a few are specifically interactive and well- 
suited for student use, three are powerful enough for systems- 
modules, one is an extention of a common scientific language 
(Fortran IV), and one is the latest (probably last) ‘super- 
language’ (PL/I). 



Table 1 . Short Summaries of Languages Discussed 

PL/I: the newest widespread compiler language — it will 

probably replace FORTRAN and perhaps COBOL. 

APL: designed by K.E. Iverson, it is an unusually concise 

and powerful programming language, best suited 
for algebraic-like problems but extendible via 
routines. 

CATO: from the University of Illinois is an extended version 
of FORTRAN IV — especially for programming 
instructional use of the PLATO system. 

BASIC: the fairly standard and very simple interactive 
language prepared originally for a G.E. 235 by 
Dartmouth College. Both compiled and interpreted 
versions now implemented. 

PIL: Interpretive interactive language modeled after 

JOSS. 

STR1NGCQMP: LISP-like features for string manipulation 
have been added to a parent language (TELCOMP) 
which was modeled after JOSS. 



Problem-Oriented Languages 

The reader may ask, why emphasize procedural features, 
especially in light of the great number of special (or problem- 
oriented) instructional languages available. Admittedly there 
is a multitude, and this very number is the first reason for 
exclusion since they have proliferated with only superficial 
differences. Se r/ ' dty, pror* ,m-oriented languages often and 
in varying degrees reiiwi u*e overpowering influence of 
individual project needs. An outside observer may detect a 
passive element in the system designer’s implementation: 

The user got exactly what he asked for, with little or no 
examination if such approaches were — in the long run — 
suited for a computer environment for instruction. For 
example, a design may mimic a frame-by-frame teaching 
machine, ignoring more splendid possibilities which the 
customer — unfamiliar with computers — would not have 
thought to ask for, but which the implementer should have 
suggested. 

Instructional Demands on a Language 
Instructional application of computer resources demands its 
own, distinct set of language powers. The lesson designer is 
concerned with display information, user input transactions, 
string manipulations, a data base which may be shared 
among users, sequence execution, data recording and mani- 
pulation for both learning strategy and teacher/author exam- 
ination, interactive terminal resources, coding convenience 
and power, supervisor resources and flexibility, and system 
accessibility. 

Procedural Elements 

It is of little use to further enumerate instances of specific 
aspects suggested in the above paragraph, especially if we are 
trying to stipulate some exact programming mechanisms 
which will be useful. Specification of functional elements of 
a language by showing possible instances of use often belies 
a vague or hastily drawn feature. To avoid overlapping 
mechanisms in a language some attempt should be made to 
invert the process: Given a set of functional requirements, 
what is some small number of language primitives which will 
meet those requirements; what more can be readily provided, 
and is the package truly adequate? Having done this for a 
good number of instructional demands, we conclude that 
the set of primitives stabilizes rather fast.* The next sections 
discuss such primitive features while concurrently describing 
in tabular form related aspects of six currently used instruc- 
tional languages. The primitives of the hypothetical language 
are, then, desiderata for contrasting these six languages. 



* Various instructional demands are listed in Appendix A. 



PROCEDURAL LANGUAGE ELEMENTS IN AN 
INSTRUCTION ENVIRONMENT 

The Basic Choice 

Even the recent deluge of problem-oriented computer 
instructional languages has scarcely disturbed the position 
of the general-purpose programming language in the field; 

abandoned by computer users, and non-procedural languages 
are usually quite inflexible. Against the limited-purpose 
convenience of the non-procedural or problem-oriented 
scheme, the general-purpose approach would pit a select, 
powerful core of basic computer transactions and possible 
logical organizations. It is then up to the installation and the 
individual user to fashion those procedures which will best 
serve them. Specific requirements of tomorrow’s users are 
often — indeed usually — unpredictable today, and this is 
where a problem-dedicated language can come to grief. 

Many general-purpose languages have found their way into 
computer instructional schemes. Iverson’s APL is a concise, 
flexible and powerful algebraic language; BASIC developed 
at Dartmouth, is almost a standard in the area of very simple, 
interactive languages. CATO is an augmented FORTRAN IV 
prepared for the PLATO System at the University of Illinois, 
PIL, the Pittsburgh Interactive Language, is an elaborate ver- 
sion of JOSS; STRINGCOMP (an extension of TELCOMP) 
is BBN’s version of JOSS with some LISP-like string opera- 
tions. For a longer list with brief descriptions see Appendix 
VI of “A Comparative Study of Languages for Programming 
Interactive Use of Computers in Instruction”, available from 
EDUCOM, 100 Charles River Plaza, Boston, Massachusetts. 



Emphasis of this Discussion 

As a starting point emphasis will be given to aspects which 
bear directly on the known problems in instructional use of 
computers. For the most part, instructional uses demand 
nothing particularly unique in interactive systems, but the 
requisite features are not generally available in older 
languages which have their origins in pre-timesharing days, 
Standard elements of procedure statements, such as assign- 
ments, control branchings and ‘IF— THEN— ELSE’s, are 
simply assumed here. Those features which are discussed we 
consider especially useful in the instructional environment. 
These elements do not comprise a universal programming 
set, but rather, shed a little light on rudimentary features 
which can be assembled into useful instructional modules. 
Appendix A lists some instructional requirements. 

Names, Data Representations, Variables and Storage 
Acquisition 

DATA. Clearly a language such as FORTRAN IV used 
alone is quite inadequate for programming most instruc- 
tional applications. One serious omission is the character 
string, which in conversational application is pivotal in 
providing flexibility and succinct code with a modest pro- 
gramming effort. Of course arithmetic is necessary, if for 
no other reason than to do file analysis and data reductions. 
List-structures would relieve programming agony in many 
simulation and inquiry applications; the pointer capability 
of PL/I provides a rudimentary feature for programming 
manipulation of lists. The pointer is intimately tied to 
dynamic storage allocation which is discussed later. Ideally 
the pointers should be relative indices, i.e., relocatable, but 
even the absolute addresses would be useful. 



Table 2. Identifiers, Acquisition Data Representations, Variables, and Storage 





PL/I 


APL 


CATO 


BASIC 


PIL 


STRINGCOMP 


Identifiers 

(lengths) 


Up to 31 j 

characters | 


Up to 6 chars, 
on 1 130 
unlimited in 
APL/360 


Standard Fortran 
Names, but also 
some reserved 
Fortran 


Usually 2 chars. J 
with restrictions 


Up to 8 
characters 


Up to 6 
characters 


DATA 

REPRESENTA- 

TIONS 

(arithmetic) 

(string) 


extensive, with 
conversions 
avail, from one 
type to another 

limited string 
lengths, and no 
pattern matching 


there are many 
powerful opera- 
tions available. A 
desk calculator 
mode exists 

could be flexible 
if built-up from 
more primitive 
elements 


special string 
features useful 
in instructional 
programming 


a scientific 
repetair, adequate 

adequate, no 
pattern matching 


adequate. A desk I 
calculator mode 1 
available 

adequate, no 
patterns 


yes with desk 
calculator 

especially designed 
for CAI string 
manipulation 


(pointer) 


yes, but are 

absolute 

addresses 


no 


no 


no 


no 


no 

. 


VARIABLES 

(label) 


yes 


standard labels 
are not mnemonic, 
but optically can 
be 


no 


no 


yes 


yes 


(array) 


yes 


yes, limited 


yes 


limited 


yes 


yes | 


(structure) 


yes 


no 


no 


no 


no 


no | 


(scalor) 


all *-> 












(event) 

(file) 


yes 

yes 


no 

not implemented 


no 

yes 


no 

yes 


no 

not on some 
implementations 


no 

? 


STORAGE 

ACQUISITION 


static automatic 
controlled based 


the allocation 
is automatic 


static (Fortran- 
like) and also 
allocation avail- 
able for re-entrant 
modules 

1 


static, but 
arrays dynamic 


automatic 


automatic 



NAMES, Identifiers should be practically unrestricted in 
length, since there is no implementation problem and the 
mnemonic worth is there. However, since one does not 
relish typing a 37 character string each time he references a 
variable so named, an alias declaration would ease life by 
equivalencing an identifier and its ‘nick name’, treating 
both as one and the same reference. 

VARIABLES. The usual variable types should be available 
along with some found now in PL/I. Aggregates of mixed 
data types form structures. Structures are very useful, 
e.g. in reading — without format — records from a file into 
core. If the structure is composed of the same data types as 
were originally placed in the file record, then the record need 
but be read into the structure and each field is instantly 
available without performing a laborious decomposition and 
conversion of the file record. Another useful item is the entry- 
point variable, allowing selected invocation of procedures. 

STORAGE ALLOCATION. Static variables are given 
storage locations and initialized at load time. Dynamic 
storage may take a number of forms: PL/L has automatic, 
controlled and based. Automatic allocations are made and 
freed with each block entry and exit as in ALGOL 60. Con- 
trolled storage is stacked and popped as requested by the 
program, and the new stack entries need not be the dimen- 
sions of the other stacked allocations. List structures are 
built up from based storage: at the time of allocation a 
variable is given the address of the block of storage acquired. 

A knowledgeable user can build whatever list-structure he 
wishes from successive allocations. 

Introducing dynamic ‘GET’ and ‘FREE’ statements into the 
language, e.g. 

GET (X,Y,Z) BINARY FIXED (31) ARRAY (1 : 12); 
we can dispense with static, controlled, based, and other 
distinctions. Dynamic variable typing and dynamic allocation 
(via GET/FREE) work nicely by stacking or popping incar- 
nations of a variable. For lists one needs a LOCATE (P,X) 
function to set the pointer P to X’s storage block, and a 
deep-reference facility for the stacked allocations of any one 
variable is very attractive. Thus: 

X = X/— 2/ ; 

sets the present value of X to the third-down in the stack. 

(The one created 3 GET’s ago.) 



GET and FREE statements determine the scope of variables 
as does the <block> in ALGOL-like languages, and to this 
extent we do not need <block>’s. Furthermore, the late 
binding times, both in variable types and storage allocation, 
make GET and FREE especially suited for an interactive 
environment. 



Files and Input/Output 

PUBLIC AND PRIVATE FILES. An instructional language 
should include facilities to read and to write in both private 
and public files. The public files will serve — among other 
things — to collect run statistics of various users or programs. 
The files should be flexible and simple to reference with 
statements in the language. The public files, for example, 
could indue 51 assorted directories which would (auto- 
matically) be used the help reference and analyze associated 
entries. Suitable file lockout and password mechanisms must 
be provided along with sequential and keyed access methods. 
An additional feature, not in the base language but built 
from it, is a flexible file editor. 



INPUT/OUTPUT. Whatever input/output facilities are pro- 
vided will probably always be inadequate. The specialized 
and varied I/O requirements of instructional programs pre- 
clude attempting to incorporate everything into the language. 
Borrowing from PL/I, we might include data-directed input — 
i.e. an input stream in the form ... A - 4, RM = 2.1, 

Z = 65.67, ... — where the variables are named in the input 
stream; and list-directed where input values 
...3,4.2,8965,3.14159,... 

are matched to correspondents in the read statement READ 
I,X,J,PI. Lastly, a formatted input should be available 
similar to the formatted reading in FORTRAN. If large 
volumes of data are transmitted, implementation efficiency 
is gained by allowing unformatted RECORD transactions — 
the raw data simply are placed in some core buffer which is 
perhaps a structure. 



Processing of strings should remain separate from input. 
Also, the language should give appropriate control over I/O 
and conversion mishaps. (See below.) 



Table 3. Files, Input/Output 



PL/I APL CATO BASIC PIL STRINGCOMP 

■ ■ y 

hm a v fa *-» on r a C‘ t !„ i.1 1 . A — 1 _ j 1 • • i i * 1 _ _ • • i , 



FILES 


an extensive 
collection of 
file types, 
including 
sequential, and 
keyed. 


files are in the 
original Iverson 
language, but 
have not yet been 
implemented 


has restricted 
system files for 
recording student 
performance 


varies with imple- 
mentation, usually 
no public files 


again, some imple- 
mentations offer 
no files. However 
is in original 
language 


can stoSre a 
program in a file 


FILE ACCESS 
PROTECTION 


lockout mecha- 
nisms provided 
in full language 






user’s ID deter- 
mines accessibility 
in some systems 




passwords 


INPUT/OUTPUT 


wide available 
range. 
RECORD- 
unformatted 
STREAM- 
data directed 
list directed 
edited 


there is a simple 
automatic l/o or 
can format into 
complex displays 


here is an example 
of l/o especially 
taylored to CAI. 
Has PLOT, READ, 
SLIDE, AUDIO, 
and similar 
commants 


simple l/o or for- 
mat controlled if 
desired 


both free and 
formatted 
options available 


free and for- 
matted plus 
PLOT feature 
for effortless 
graphs 



Statements and Modules 

PROCEDURES. Compound statements and ALGOL-like 
blocks provide conceptual convenience for the programmer. 
Procedures types should include both internal, i.e. sharing 
names with other program elements, and external — com- 
piled separately. Generic procedures allow standard calls for 
syntactically distinct sets of arguments and are quite useful. 
Co-routines are procedures which have a ‘restart’ capability 
as distinct from the usual call. The checkpoint feature 
which stores crucial activation information of a co-routine 
should be optionally controllable; in this manner a pro- 
cedure could be ‘saved’ at various progress points and later 
begun selectively at any such ‘save’ point. Not surprisingly, 
synchronized procedures are easily written if each is a co- 
routine calling the other. 

TASKS, PARALLELISMS. PL/I has “tasking”, where the 
main procedure spawns a procedure which then runs asyn- 
chronously from the parent. As an example, some aspects of 
a student’s answer may be given a thorough scrutiny by a 
task while the main procedure continues the usual dialogue 
with him. Since both main program and task may be await- 



ing execution or executing in parallel processors together, 
some method of synchronization may be required if the 
process is to rejoin paths. In PL/I the WAIT (EVENTNAME) 
functions in this capacity, holding execution until the task 
called EVENTNAME is done. The statement AND could 
also effect a fork in the control path, to be followed by a 
JOIN which signals confluence. For paths which do not 
rejoin, the command ALSO would do. 

Interrupts 

PL/I introduced to the general computing community 
controllable interrupts for error and global program control, 
and there is little reason to abandon this valuable feature. 
Furthermore, the idea of SIGNAL enables the programmer 
to simulate an interrupt and so capitalize on the global 
properties of the interrupt mechanism. The block of code 
executed with a given interrupt should ideally allow 
1) arguments, 2) examination of crucial variables, and 
3) optionally allow resumption at or beyond the point of 
interrupt. Unfortunately these stipulations may lead to 
undefined quantities; it is not clear they are worth the 
effort. 



Table 4. Statements, Modules, Extensions, Code 



CATO BASIC PIL STRINGCQMP 



STATEMENTS 

(form) 


FORTRAN-like 


have strong 
algebraic flavor 


FORTRAN plus 
extra forms for 
CAI 






i. 

i 


BLOCKS 


ALGOL-like 
with local 
variable 


internal 
routines may 
have local 
variables 


program unit 


program unit has 
overlap feature 
via CHAIN 


PART’S 

function as blocks 


PART 


PROCEDURES 


internal, 
external, 
generic, and 
recursive 




fortran 

convention 
with some 
reserved words 


has simple and 
compound 
functions, 
internal sub- 
routines 


DO PART 


functions are 
only single 
statements 




INTERRUPT 

CONTROL 


via ON . . , 
statement may 
signal interrupt 
via SIGNAL 


no 


may checkpoint 
program so other 
programs can run 









EXTENSIONS 


pre-syntactic 
text macros, also 
run-time mapping 
macros 


via routines 
and functions 


could probably be 
further extended 
as has been 
already 


not much hope 


perhaps the PART 
element 




RE-ENTRANCY 


yes? 


yes? 


yes 


yes 


yes 


no? 


RECURSION 


yes 


j yes 


1 


yes 


yes 

J 


1 was designed to 
| teach recursion 



Other Points 

A useful feature is interface facilities with other languages, 
especially an assembly language. Because macros offer exten- 
sible features which allow fruitful language experimentation, 
the language should incorporate a macro facility. For 
example, template macros allow disguised routine calls while 
checking arguments for correct syntactic type, as in: 
WHENEVER A OR B IN STRING, RANGE IS (5, 7) 
for the routine SNORT (A, B, STRING, 5, 7) by using the 
macro definition 

TEMPLATE: ‘WHENEVER’ 1/real/ ‘OR’ 2/real/ ‘IN’ 3/string/ 
\ RANGE IS ( 6 ,4/int/‘,‘5/int,‘)‘TEMPEND; 

BODY: ‘CALL SNORT (T, ‘2’, ‘3’, ‘4’, ‘5’)’ BODYEND; 



The macro checks the types of its arguments and then 
substitutes them into the body text. All text is enclosed in 
single quotes. A further example may be found in Appendix 
C. Additional desirable implementation are 1) re-entrant 
code, and 2) recursive procedures. 

Summary 

Obviously nothing mentioned in this framework is really new. 
Indeed, in instructional computer usage — like so many other 
applications - there is a need for a wide variety of language 
powers and almost all of them have arisen before in some 
other application context. One simply borrows what is 
needed. In this case, mose of the lending was by PL/I. 
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THE LANGUAGES 

Among the languages considered, BASIC, PIL, AND 
STRINGCOMP are all similar interactive languages which are 

1. easy to use 

2. simple to learn 

3. fast for debugging small programs. 

For student programming and learning, or for a modest 
research effort, these three languages definitely have some- 
thing to offer. STRINGCOMP and PIL are stronger in string 
operations — something to keep in mind. 

An actual (production) teaching system should use a com- 
piled language for running efficiency. Within this class, 

PL/I is one of the newer and more powerful, borrowing as 
it does from ALGOL and COBOL while trying to look like 
FORTRAN. And the vast spectrum of programming elements 
available — the data types, for instance — lend to it an attrac- 
tion unmatched by any other in this group of six languages. 
For one who wants to extend a current language for instruc- 
tional use the details of CATO — viz. intended FORTRAN — 
should interest him. Here the base language is very well 
known; the programmer need spend only that time necessary 
to learn the new features before going to work. Furthermore, 
since FORTRAN routines are compatible in the system, a 



system development could be broken into segments, some 
of which are pure FORTRAN and therefore accessible to 
regular FORTRAN programmers. 

Elegance is APL’s calling card. It is concise, powerful and 
extracts some effort from the would-be user. Having worked 
into the intricacies, one might very well claim — as many do 
— that the effort is well spent. Several instructional projects 
have used APL and the new implementation for the IBM 
system/360 series machines may further this number. 

None of the languages in the group of six share all the pro- 
gramming elements of the hypothetical language. The most 
glaring absence is that of suitable file support. Curiously 
enough, files are discussed under various guises in the 
instructional language literature, but most proposals are 
limited in scope and often awkwardly implemented. Another 
plea might be extended for programmable interrupts — they 
are so useful — and a good macro facility. Macros written by 
each user enable him to converse in concepts relevant to his 
field. 

IN SUMMARY, why not an instructional language which is 
general-purpose, flexible, and moderately extendable? 
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APPENDIX A 

Some Requirements in Instructional Languages (key phrases) 

DISPLAY INSTRUCTIONAL MATERIAL 
assemble material, display characters, display graphics, 
display audio, time display. 



VARIABLES - TYPES 

scalar 

label 

array 

file 

structure (mapping templates, cf. storages references below) 



ACCEPT STUDENT RESPONSES 
accept characters, error correction by student, accept 
graphics, accept audio connect terminals, time response, 
interrupt response. 

PROCESS STUDENT RESPONSES 

process characters, process numerics, process algebraic, 

process graphics, respond to unrecognized. 

IDENTIFY LOCATIONS IN INSTRUCTIONAL SEQUENCE 
label a statement, identify by condition, establish a restart 
point, branch to label, branch to condition referenced, 
branch to subroutine, return from subroutine, conditional 
branch, select from list. 

RECORD AND MANIPULATE DATA 

student data, program data, store data for later summary, 

computations. 

STUDENT LEARNING TOOLS 

desk calculator, algebraic language, text processor, simulation 
language, information retrieval. 

CODING CONVENIENCE 

abbreviation, assign attributes, model conventions, other 
available languages, implicit branching, automatic data 
summary, precoded procedures, comment or remark. 

AUTHOR OPERATIONS AND UTILITY ROUTINES 
preparation of program, format of program, translation, 
automatic assistance, manual assistance storage author edit. 

PROCTOR OPERATIONS 

register users, restrict use, on-line assistance, records 
handling error identification. 

Other Factors of Interest 
SYSTEM OPERATIONS 

translator, operating system, processor and storage, terminals 
and communication, automatic recovery, retrieval of lost 
material. 

TYPICAL USE AND REFERENCES 

subjects, students, levels, strategies, applications, reports of 

use, stated purpose, a manual documentation. 

APPENDIX B 

Elements from a Hypothetical Instructional Language 
(key phrases) 

DATA REPRESENTATIONS 
arithmetic (integer and floating point) 
string (character and bit) 
pointer (relocatable so that lists can be filed) 

IDENTIFIERS 

mnemonic (unrestricted length) 



STORAGE ALLOCATION (typing and allocation are 
dynamic) 

dynamic typing/ allocation via GET/FREE 
allocations of a variable are stacked with deep-stack 
referencings available (i.e. can get ‘old’ values) 

STORAGE REFERENCES 

dynamic typing and allocation of variables, along with 
selectable mapping functions for aggregate variables provide 
a flexible control of storage. Auxiliary functions such as 
LOCATE (P,X) to establish a pointer P to variable X, DEPTH 
(X) to return a count of available allocations of X, and 
e.g. TEMP3@X for “overlay”' referencing of X by tem- 
plate 3, are among the language features. VARTYPE (X), 
to return the type of a variable, is also necessary. 



FILE FACILITIES 
sequential, keyed 
public and private 

access control: passwords, lockout mechanisms 

GENERAL I/O 
Directed (formatted) 
data-directed 
list-directed 

edited (via a FORMAT statement) 

Undirected (no conversion of any sort) 

Appropriate interrupts should be accessible in the language 
(e.g. conversion errors) 



MODULARITIES 

compound statements 

omit block structure 

internal/external procedures 

co-routines and necessary checkpoint statements 

parallel executions 

begin parallel paths (fork), join latter 
fork and no need to rejoin 
wait for all confluent control paths 
e.g. ON OVERFLOW DO; . . .; END; 
pseudo-interrupt, e.g. SIGNAL 
(ERROR) 



AND (label) 
ALSO (label) 

JOIN 

interrupt control 
signal 



MACROS, EXTENSIBILITY 

interface features whh other languages, especially assembly 
language template macros for subroutine calls (disguise them) 
perhaps the language could be made available for computa- 
tion during macro-expansion (a two pass interpretation), in 
this case the macros could define blocks, and generally carry 
a much heavier load. Translation-time scratch files would 
also be required so macros could bookkeep and communicate 
among themselves. 

ADDITIONAL FEATURES 
re-entrant code (everything) 
recursion (an option) 



0 



23 



APPENDIX C 

An Example of the Use of Macros 
PROBLEM: 

An author wants to build a simple file system for drilling students, and he chooses the file format below, which is much 
like a SNOBOL statement form. 



file 


key 


question 


answer 


answer 


true 


false 


record 




text 


pattern 


text (if wrong) 


key 


key 



To facilitate building the file, the author constructs an input statement which automatically checks his data and writes it 
into the file called SNOWFILE. The input lines which the author composes have the form. 

LINE/34/this is a question/‘this’, ‘question’/not a question/35/34. 
. 

Each such above entry causes the macro (which the author wrote) to be executed. This macro is given below: 



TEMPLATE: ‘LINE’ *l/string/7‘*2/string/7‘*3/string/7’ 

BODY: IF *I=‘STOP’ THEN DO; WRITE(TELETYP) ‘END FILE BUILDING?’; 

GOTO ENDOFTHING: 

END; 

ON TRUNCATION DO; WRITE(TELETYP) ‘YOU RAN OVER A FIELD’; 

GOTO ENDOFTHING: 

END; 

KEY :=*I;QTEXT :=*2;ANSWPAT :~*3;ANSTEXT :=*4; 

TKEY:=*5 ; FKEY :=*6; 

WRITE(SNO WFILE)RECORD(KE Y)KE Y,QTEXT, AN SWP AT , AN STEXT , TKE Y,PKE Y ; 
ENDOFTHING: /* THIS IS A NULL, STATEMENT FOR EXIT */ 

BODYEND; 



Notice that the macro expands into no text when interpreted. Instead, it causes transactions with the file SNOWFILE. 

Having completed the file our hypothetical author would now like to run the exercises. He writes two macros with the 
calls: 

and NEXT QUESTION; 

PROCESS ANSWER;. 



These are the two main statements in the author’s program. A complete drill might be as below: 



PROCEDURE: OPTIONS(MAIN); /* DECLARATIONS FOLLOW */ 
GET KEY CHARACTER^), QTEXT CHARACTER(80), 

START: KEY:=001 ; /^INITIALIZE KEY TO FIRST KEY*/ 
CONTINUE: NEXT QUESTION; /* MACRO CALL HERE */ 
PROCESS ANSWER ; /* ANALYZE THE ANSW*/ 

IF KEY *= ‘STOP’ THEN GOTO CQNTINUE;./*DQNE?*/ 

END; 
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The macros which the author wrote are given below: 



macro 1 TEMPLATE: ‘NEXT’ * 1/string/ ‘QUESTION’ TEMPEND; 

BODY: ‘ READ(SNOWFILE) RECORD(KEY) INTO (QTEXT,APAT,ATEXT,TKEY,FKEY); 

WRITE(T ELET YP) QTEXT; /* ASK THE QUESTION */ ’ BOOYEND; 
macro 2 TEMPLATE: ‘PROCESS’ * 1/string/ ‘ANSWER’ TEMPEND; 

BODY: ‘ READ(TELETYP) ANSW; CALL ANSINTERP(ANSW , ,APAT ,SW); 

I* ANSINTERP WILL CK THE ANSW AGAINST THAT IN THE FILE ANSW PATTERN / 
IF SW=‘F’ THEN DO; KEY:=FKEY; /* WILL BRANCH TO FALSE* ANSWER PLACE*/ 
WRITE(TELETYP)ATEXT ; /* WRITE OUT ANSWER */ 



END; 

ELSE KEY:=TKEY; /* ELSE GO TO TRUE*ANSWER BRANCH */’BODYEND: 



Notice that 
other spurio 

call, e.g. fro:.. v - - 




o 
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A COMPARATIVE STUDY OF LANGUAGES 
FOR PROGRAMMING INTERACTIVE USE OF 
COMPUTERS IN INSTRUCTION 

(Excerpts from the final report of an EDUCOM 
study funded in part by the Office of Naval Research 
during 1968 and available from EDUCOM, 100 
Charles River Plaza, Boston, Massachusetts) 

Karl L. Zinn 



J. COMPARISON OF LANGUAGES 

A. By common aspects 

The forced juxtaposition of two or more languages whether 
by a list of aspects, characteristic samples or measures of 
author performance and satisfaction cannot help but improve 
the capabilities of each for common purposes. 

The comparison of languages by common aspects emphasizes 
similarities by presenting together the way in which various 
languages accomplish the same function. It has prompted 
some languages designers to fill in a few blanks in the columns 
describing their languages, that is, they have provided their 
users with capabilities described for other languages in the 
table. A means for translation from one language to another 
is also implied. 

A summary table arranged by common aspects cannot be 
complete and free of error: the languages are changing 
rapidly; the designers are slow to provide current documen- 
tation; first-hand programming experience in each language 
is not possible. Different approaches to summarization 
favor one language or another; different approaches to 
instructional use of computers require essentially different 
language characteristics. 

Languages explicitly intended to do different tasks should 
be described by different sets of attributes in different 
tables. 

When making decisions about languages and systems the 
relative weighting of various criteria must be determined by 
each project or user upon considering: a) the age and back- 
ground of the student, b) the relative importance of research, 
development, implementation and operations, c) the interest 
of the project staff in general system characteristics versus 
programming languages versus instructional materials, and 
d) the availability of funds and of a general-purpose system. 

Some standard format or common notation is needed for 
writing an individualized description of each language. In 
order to communicate with potential users of computer- 
based systems, a notation for description should be readily 
interpreted by those who have little experience with 
languages and systems. 

B. By characteristic samples 

Each language has unique features, and any small number of 
test programs will favor one or another. It is difficult to 
represent the capabilities of any language in a few pages of 



sample programs. 

Without agreement on optimal strategy for computer-based 
instruction, any small number of test programs will favor one 
or another kind of language. In fact, it is difficult to represent 
the capabilities of any language in a few pages of sample 
programs. Many of the languages under study continue to be 
changed, and the samples obtained one month may not be 
characteristic of what is being done with the language six 
months later. 

C. By usefulness: effectiveness, convenience, 
reliability, etc. 

A major problem for evaluation is the definition of a 
measure of accomplishment which voids reliance on how 
long the student spends at a computer terminal. Studies of 
cost-effectiveness could proceed if the experts in some 
subject area would agree on measurement of concepts 
acquired or skills perfected, which would apply throughout 
the domain o'f concern. There is little hope for relating results 
from use of new media to conventional-instruction- 
equivalents because curriculum designers will and should 
exploit the new medium to improve aqd expand the content 
and skills taught, and will use the occasion of revision to drop 
some material which is obviously useless. 

Criteria for measurement should be more objective. Words 
such as reliability and flexibility are used as if everyone 
agreed on what they mean. In some cases very different 
measures of the implied concept have been employed. It is 
not necessary that all agree upon any single definition for a 
term or unique measure for a criterion, but the various uses 
must be made explicit. 

Journals should adopt a firm editorial policy which requires 
clarification of the referent or measure ‘power’, ‘elegance’ 
and other such terms when used in published reports. 

II TYPES OF LANGUAGES 
A. By user 

(2.) Those involved in operational uses of computers for 
instruction in the schools should not pay for computer time 
and equipment that is not part of the immediate task. A 
system designer may increase operating costs as he adds on 
various optional features. 

(4.) Staff on a curriculum development project require 
convenience and low error rate for writing and testing the. 
material. This is not the same as economy and convenience 
for student use day-to-day in the schools: terminals may be 
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more expensive, the speed of compilation of new materials 
more rapid, priority is given to revision of materials, etc. 

A project working on development of system features and 
language characteristics should invest in flexibility. Further- 
more, the curriculum writers who choose to work with such 
a project must be willing to give up some convenience for 
the sake of experimentation, eg., learn new language 
features frequently, modify or discard programs which used 
outdated procedures, accept errors and unreliability in the 
system, etc. 

B. By strategy of curriculum development and/or 
description 

1. The most straightforward approach to serving the needs of 
an author may be to provide a format into which he places 
elements of the curriculum. The computer program 
successively presents the question frames, provides a hint 
when the student asks for it, provides the right answer when 
needed, and records performance data for later inspection by 
the author of the exercise. 

2. Frame-oriented description of testing or instruction is a 
kind of computerized programmed instruction. The similarity 
of the code and conversation to a programmed text is 
apparent. In fact, translators have been written to accept 
linear (or simple branching) programmed text and derive 
CAI interaction with a student. 

More author hours have been invested in the computerization 
of programmed instruction text than other instruction modes. 

“Frame-by-frame programming with an author language is on 
the way out. A few years from now less than one tenth of 
any computer-based course will be programmed by an author 
or his technical assistant in languages such as 
COURSEWRITER and PLANIT.” 

Materials which now are prepared using these languages 
include a number of pre-defined frames or “paragraphs” in 
an instructional sequence, each one written in some detail 
and on an individual basis. This approach cannot succeed; 
too many writers are required to generate enough instruc- 
tional material in this format to provide for the needs of our 
increasing educational populations and to justify large scale, 
computer-based systems dedicated to this mode of 
operation. 

3. Task-oriented notation. Whenever one or more authors 
have a singular instruction task, determined in part by 
content and in part by instruction strategy, a special 
language or dialect may.be useful. Eg., PLANIT is particularly 
suited to tutorial and problem solving in statistics; MENTOR 
was developed at Bolt B ,ranek and Newman (BBN) 
especially for “socratic dialogues.” 

4 . Procedure-oriented languages. All of the languages or 
notations in the first three categories had to be programmed 
for the computer in a regular computer language which could 
be interpreted by the machine. Any of these languages could 
have been used directly, but some are especially convenient 
for writing procedures for interactive use on a computer, or 
for conversational instruction in particular. 

The author adopts a logic or pattern or strategy which can 



be conceptualized in a procedure statement. The 
programming of the logic may have been done by the author 
or by a programmer experienced with the system. The author 
then applies this and other strategies to curriculum files of 
indefinite size. 

Increased use of procedure-statements and (separate) 
curriculum files will be beneficial for the field, and increasing 
use of computers in large curriculum projects will require this 
approach for economy. 

Procedure-oriented languages are for computer programmers 
and for educational technologists specializing in computer 
applications; these persons should produce the user-oriented 
languages or data formats which maximize convenience of 
the curriculum expert. 

It would not be advisable to promote only development of 
generative procedures. Some instruction packages have been 
demonstrated to be useful as they stand; and more is yet to 
be learned about computer aids from pushing the limits of a 
frame-by-frame approach to programming. However, a 
significant proportion of resources for research and develop- 
ment will be put into more flexible approaches to designing 
learning environments. 

One could have too large a library of strategies and too much 
individuality among students and topics for standardized 
techniques to be useful. 

Subject areas will not see much development of prescriptive 
curriculum for individualized instruction (controlled by the 
computer program) until instructional objectives are rather 
well defined in terms of practice in using facts, concepts, and 
simple skills which can be described and administered by 
standard procedures. 

The practical application of standard procedure programs and 
generative techniques applied to curriculum files on any 
specific subject area or training situation raises the following 
questions: How are information structures to be described 
by the subject expert and stored in the computer for use in 
such procedure statements? How are materials to be 
assembled according to general rules? In other words, do we 
know how to process language in this general way? Can 
student answers be processed except by empirical procedures 
derived by the author from experience with student responses 
to each individual question? 

How do we process input from the student in some general 
way which determines a suitable reply? Can we identify 
patterns or sequences which indicate certain treatment for 
the student on succeeding learning experiences? 

5. Interactive student programming. If an on-line problem 
solving language is suitable for simulation and model building, 
then that language certainly is of interest to designers of 
computer-based learning exercises. First, the subject expert 
may want to build models on which to base games or 
simulated practice for students to try. Second, he may wish 
to guide some students through revision of the models and 
construction of new ones. In general, he wants to show 
students how to use the computer for information processing 
in his discipline; as lesson designer he might produce a 
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“mentor’! which advises each student on how to get maxi- 
mum value from the computer as a problem solving and 
scholarly aid. 

Given limited computer resources and very few expert 
personnel to design learning exercises, projects are advised to 
use the computer more as a learning tool than a presentation 
device. The relative benefits are apt to be considerably 
greater when the computer does things which could not be 
achieved in other ways; students of reasonable study skill 
can learn rather well from textbooks, workbooks, and other 
non-computer learning formats. 

The most significant contribution of simple, interactive 
programming languages may be through increased student use 
of computers for problem solving and scholarly endeavor on 
individual initiative. 

Much of the enthusiasm for conversational computing 
languages may relate to non-essential features; quick response 
and understandable diagnostics can be provided'in batch 
systems. Now that commercial services are being offered to 
(and purchased by) public schools, it becomes increasingly 
important to isolate the essential contributions of interactive 
programming languages, and to determine effective cost 
conditions. 

The computation facilities of the interactive languages are 
useful in many computer-based learning exercises, and 
convenient computation is conspicuously absent from nearly 
all languages specially designed for computer-assisted instruc- 
tion. When an author uses an interactive language the 
program is more accessible to the student. This increases the 
opportunity for a student to become involved in the re- 
design of an exercise, and this is especially important when 
the curriculum writer has incorporated mathematical 
models or simulations. 

The problem-solving mode will be over-valued and misapplied, 
as was COURSEWRITER five years ago. However, more 
instructional materials of significance are likely to survive in 
this mode in the next five years, than have been seen in the 
computerization of programmed instruction in the last five 
years. 

Ill UNIVERSALITY OR UNDERSTANDABILITY 

A. Universal language 

Strong forces will be encountered against standardization, 
but one common language is not the important goal. Because 
of the great variety of purpose and process in instructional 
programming, less progress will be made toward standards 
than in business or scientific programming. 

B. A few common languages (eg, one of each type) 
COURSEWRITER will continue to be used, as FORTRAN 
continues to be used, not because of general utility, but 
because it handles a few jobs well, it is available on a number 
of different machines, and users have some hope of transla- 
tion from one dialect to another. 

Five years from now neither COURSEWRITER nor PLANIT 
will be the standard, although there will be some efforts in 
that direction. Hopefully some guidelines will be achieved for 



talking about language, and more important, for describing 
and documenting instructional procedures. Such a communi- 
cation medium will promote design of more reasonable 
instructional tasks, and serve also as a significant tool for 
advancing instruction research and strategies of curriculum 
development. 

C. Translatability 

New languages and systems will have greater capacity for 
translation of instruction programs from present 
programming languages in which they were implemented. 
Translatability is possible without imposing any restrictions 
on innovative ideas for language or strategy. 

D. “Publication” language for documentation and 
communication 

Documentation has two main functions, that of information 
transmission and work simplification. It transmits informa- 
tion to potential users concerning: (1) the contents of the 
instructional program; (2) the proper use and application of 
the program, and (3) how the program was constructed. It 
simplifies work by: (1) enabling the user to find actual or 
potential trouble spots; (2) assisting the user to eliminate 
problems which may arise, and (3) simplifying revision. 

Representation of a procedure statement for a curriculum 
expert not accustomed to computers requires an approach 
different from standard flow charting. 

In most cases the essential information about a computer- 
based learning exercise can be derived without executing the 
program; careful study of proper documentation should 
provide all information about the materials and logic short 
of operating system characteristics. One obtains relevant 
information more efficiently through organized exploration 
of the program description than through reading individual 
records of student-machine interaction or through blind 
searching on-line at a student station for the eventualities 
for which the author has provided coding. 

Actual experience with a computer-delivered exercise may be 
an important contribution to understanding and evaluating 
an instructional unit, especially if certain knowledge and 
technique are supposed to unfold or develop during the 
learning experience. An important component of some 
learning experiences is effective, that is, success depends on 
an impression or feeling of pleasure, satisfaction or possibly 
surprise. Negative experiences might also be identified in on- 
line experience more readily than in a statement of 
specifications. 

IV GENERALITY AND SCOPE OF INSTRUCTIONAL 
SYSTEM 

A. Hardware limitations 

Uses of auxiliary memory for updating formatted files of 
student records and making decisions in real time on the 
basis of certain aspects of the data require direct access to 
specific portions of the information. It is disappointing to 
find the disk and drum storage on conversational computing 
systems used in a tape-like fashion instead of as the direct 
access file devices they really are. 



B. Software limitations 

When the subject expert and educational technologist 
become distracted from their real purposes by the peculiari- 
ties of current computer systems and programming 
languages, work should leave the computer for a time until 
the essential parameters of the learning situation are deter- 
mined. If specifications for human tutoring are prepared as if 
for a more sophisticated computer system than now available, 
techniques developed off the computer will more readily be 
adapted for computer implementation later. 

A broadly conceived instruction system probably should 
begin with a general-purpose system and add facility for 
moving from the tutorial mode into other user sub-systems 
and returning when an exercise is completed. Some authors 
need to maintain contact with the student through some 
means of monitoring his work on a problem, to bring him 
back to the tutorial mode because of elapsed time, number 
of problem attempts, or even an anticipated error which 
requires special attention. 

Instructional systems should incorporate other programming 
capabilities which can be used by both author and student. 

In addition to simple computational aids, some lesson 
designers will want to provide an algebraic language, a text- 
processing language, a model-building or simulation language, 
perhaps a specific system or model written for student use, or 
information organization and retrieval capability. 

A task-oriented language may serve well those purposes for 
which it was originally designed, but authors in other areas of 
study will need to have the translator adapted for their own 
special needs. For a long time, users who seek ready-made 
specific aids and generality at the same time will be 
disappointed. 

Revision of materials should be encouraged by automatic 
summarization and selection of data on computer cost and 
student time as well as student performance to be put before 
the author. 

Capability for processmg quantities or algebraic expressions 
typed by the student is useful. The author may wish to 
attempt to match a number within numerical limits, or as an 
integer multiple or negative inverse of the anticipated 
number, or match values in spite of an error in units. Some 
languages already provide for evaluation of an expression by 
the translator; but in some instances one wishes to recognize 
an expression as equivalent to or in some definable way 
different from the expression anticipated by the author. 

C. Cost limitations 

Techniques for preparing curriculum files must be more 
powerful in the sense of fewer hours required of the subject 
expert to write and revise materials which achieve the objec- 
tives intended of the learning experience. Authors cannot 
often afford the luxury of individually shaping or tailoring 
each line of text in each frame for each kind of student. 

It is today cheaper, and in some instances perhaps more 
convenient, to handle some desirable translator features 
manually with clerks and writing assistants. The next 



important step is careful development and evaluation of 
language features which adapt to the needs of authors and 
subject areas. 

Conversational languages emphasize convenience, and some- 
times require considerable additional cost in computer time 
during execution. The number of operations for inter- 
pretation of a symbolic program is always greater than for 
execution of a program already compiled into machine-level 
statements. Of course a user may be willing to pay more for 
execution if his results will be available immediately and 
without complication, along with quick diagnostics and 
opportunities for changes in the program at stopping points 
throughout. 

D. Design procedures and/or limitations 
One way to extend a language to handle additional applica- 
tions is to provide iinkage to other programs. No one 
language now available can handle the variety of applications 
efficiently, and some useful subroutines may already be 
available in other languages on the same system. The major 
problems seem to be: 1) transferring data, 2) returning 
control to the calling program, and 3) leaving the user in 
control in spite of program or system errors. 

Facility for definition of functions should be extended to 
provide for definition of a) character operations as well as 
numeric ones, and b) distributed operators which apply 
throughout one or more statement lines. The latter would 
allow for definition of new operations with convenient 
formats for specifying answer processing. More than one line 
should be permitted in the definitions, and the possiblity Of 
an operator being distributed among two or more variable 
names must be allowed in the parser. 

The problems with extending a language through definition 
of new operators and statement types concern the internal 
representation of the language, simple rules for describing 
new features, and the ability to recognize operators 
distributed throughout a list of variables even on more than 
one line or program statement. 

New features defined as functions for execution as needed 
must be reinterpreted each time the function is used, and 
little economy of execution results. The ability to compile or 
assemble a routine, link it to the interpreter, and specify its 
execution in a statement form natural to the user will 
increase the convenience of conversational computing 
languages while making certain information processing 
operations more economical to perform. 

One way to accomplish some economic advantage is to re- 
assemble the interpreter, adding the new statements, 
functions or operators to the language. This delays availability 
unless an informed system programmer is always at hand. 
Reassembly for one user also raises some questions of 
proliferation: should he then have his own special version; do 
changes in the basic compiler take effect for everyone? 

Variety and flexibility in programming capability of an 
instruction system are not necessarily incompatible with 
economical operations. Early decisions by system designers 
about specifically what is needed by users inappropriately 



limit the scope of applications. 

It is not obvious what the elements of programming should 
be. The basic statements and operations need be elementary 
enough to permit building the variety of processes desired by 
programmers. However, high level commands should be 
assigned to frequently used routines constructed by 
programmers in a way that the syntax can be readily used by 
curriculum designers. 

The potential author who does not have access to a specially 
designed educational system, or the school which wishes to 
gain some experience without large investments in space, 
leasing arrangements or committed user time, will find the 
rental of one terminal for limited use of a general-purpose, 
time-sharing system to be a satisfactory interim arrangement. 
Through suitable modifications, some of these languages and 
systems could serve most of the needs of instructional 
applications. 

Interactive programming on a general-purpose system is not 
likely to include the proctor operations and other systems 
support which may be important to educational experi- 
ments and operations. 

V. INTERACTIVE MODE CONTRIBUTIONS 

The essential contribution of interactive programming must 
involve responsiveness of the system which holds special 
benefits for the casual and infrequent user. He may be well 
advised, when unsure of the proper syntax, to try various 
likely ways until the interpreter accepts one and does what 
he intended. Better yet, the processor should tell him what 
form to use the first time an uninterpretable statement is 
entered, or refer him to the section of a reference manual 
which is likely to explain away his confusion. 

Interactive programming languages incorporate aids for 
program testing in a very natural way. The same statements 
with which stored programs are written can be used as direct 
commands to print the values of selected variables to see 
what went wrong, assign new values to test other parts of the 
procedure, and resume execution with any line or segment 
of the program. 

If diagnostics, provided at the moment, and backed up by 
references to readily available literature, can relieve the user 
of conQern for the means to describe his procedure, he will 
give more attention to solving the problem. A shorter 
elapsed time between problem definition and solution, and 
the time savings attributable to. continuous working sessions 
provide another bonus. 

A cleverly written processor should have some auxiliary 
memory and decision rules which generate special user 
assistance. A rather deep search for the locus of a syntax 
error and some attempt to interpret the intention of the 
user in spite of ambiguity has already been mentioned. 

The processor might also keep some record of the types of 
errors made by the current users and, according to rules set 
up by the designer of the training and support system, 
initiate some conversation with the user in order to clear up 
his probable confusion. 



The processor might save previous versions of statements and 
segments which are replaced, until it becomes less likely they 
will be used again. Already the user can store alternate 
routines under different names, using one or the other during 
testing, until he feels ready to discard all but the most 
effective one. A careful design might automate part of this 
process, increasing the power available to the initial or 
infrequent user. 

There could be considerable convenience in a memory 
feature 'which reduces the distinction between the indirect 
mode of stored programs and the direct mode of commands. 
For example, statements would be executed as entered, with 
changes in sequence (branching) requiring confirmation by 
the user. The translator would monitor selected variables and 
indicate when they exceed acceptable ranges defined by the 
user. Entire conversations would be saved so that part of an 
earlier attempt at problem solution could be retrieved for 
examination; a program listing could be extracted for 
execution or permanent storage. Alternatively, the user could 
request an automatic but selective saving of alternate 
versions of statements or short routines. 

Incorporation of training into the use of an interactive 
language introduces some problems. One consideration is the 
time required for typing full diagnostics, remedial material or 
tutorial lessons. A training version of a processor could be 
availabb in addition to the regular operational version. A 
training mode could be incorporated throughout the regular 
processor if a convenient convention is provided for request- 
ing additional clarification or diagnosis as needed and no 
more. For example, the documentation for the language 
might be stored in the computer in a tree structure referenced 
by pointers in the translator. Whenever the scanning routine 
ran into difficulty, it would point to some place in the (on- 
line) manual to which the processor would refer the student. 
Additional indications of confusion would cue increasingly 
detailed responses from the processor to some reasonable 
depth. If the interactive mode is fully exploited, and the 
manual reasonably arranged, the availability of help should 
impose no delays or distractions on experienced users. 
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DISCUSSION 

Captain Huggett said he felt that this was perhaps the most 
dependent of the sessions since the design of a CAI system 
and its attendant software should be the outcome of other 
wider factors starting with a statement of the education or 
training goals, a knowledge of the capabilities of the popu- 
lation that is required to achieve them, a knowledge of the 
ways in which the material might be learned, and so on. 

The position papers had fully recognised this point and it 
was interesting for him to compare the general aims, 
consciously arrived at or not, of a number of centres where 
computer based learning facilities existed and he instanced 
the case of PLATO at the University of Illinois where the 
aim appeared to be to produce a community service in 
which many different approaches to learning could flourish. 
He remarked on the very stimulating environment tins 
eclectic environment produced. This sort of aim would 
lead to a very different specification of hardware than aims 
of a more specific nature. 

The design of CAI systems is limited by the factors that 
limit communication in any form. Fundamental problems 
of size and cost are posed by the data demand on the 
system, the rate at which it may be supplied and the costs 
of transmitting this data. A range of terminal devices had 
been listed with broad assessments of their capabilities and 
cost. Captain Huggett pointed out the imbalance in the state 
of terminal development. Thus the ways in which the stu- 
dent may respond to the system are narrower and less 
flexible than the ways in which the system “talks” to the 
student. This restriction would not impair certain types of 
learning but there are many that it would. He also pointed 
out that students also expect machines to have machine like 
characteristics and there are indications already that even 
quite modest delays in machine response are unacceptable. 
The system employs a teaching strategy but this must be of 
a sort which is in harmony with the way the student wishes 
to learn. Dr. Adams put it in a particular way when he said 
that “The designer’s conception of the user is implicit in the 
operating system of the machine and thus has a strong struc- 
turing effect on him (the user)”. This seemed a very import- 
ant consideration that ought to be discussed. Those author 
languages that he had seen reflected the designer’s view of 
the user but they appeared not to have been conceived as 
entities and the guidelines adopted early on sometimes 
proved to be an embarrassment later on when developments 
were necessary to a language that had become too big and 
complex to be altered without being rethought. CAI was 
characterised by its highly interdisciplinary nature and the 
success of CAI ventures would depend on good managers 
who could direct and blend these disciplines severally 
developed in others. 

Captain Huggett ended his summary by saying he would like 
ic hear discussion on the degree to which CAI systems 
should be developed to simulate/emulate human student/ 
teacher interaction, the degree to which the universal author 
language should be developed, what learning model should 
be used to define language, and the question whether com- 
paratively free unfettered hardware development would 
throw up these answers anyway. 

Professor Pask said the author language should reflect a 



learning model but the trouble was there were many of them. 
The ‘black box’ model was very convenient and would work 
for simple highly structured tasks but it was not a larvning 
model and could not be used to predict. He emphasised that 
teaching was control of the learning process and that be- 
cause of the complexity a complete CAI system would be 
a hybrid of different forms of control system and that a 
possible approach was to try partially to isolate subject areas 
and to try to optimise with respect to the several parts. 

Professor Mitzel commented on the ad hoc nature of work 
in the United States so far and the fact that CAI needs 
theoretical definition. He called for a study of educational 
objectives and pointed out that examinations test what each 
student has obtained from some set of experiences but if 
this was looked at in terms of his later achievements a 
different pattern of education might evolve. He also touched 
on the nature of a conversational mode between student and 
machine. Did this involve only the exercising of options by 
the student or was it possible for the teacher to retain some 
degree of control. He saw-the machine being used to attend 
to the delivery and continuity of material leaving the teacher 
free to deal with the affective issues. 

Miss Ash welcomed this turn in the discussion and said that 
care was needed to prevent too much emphasis on fitting the 
student to the system and there must be more on maMng 
the system flexible enough to fit itself to the student. She 
called for a terminal development programme which would 
accept freely constructed student responses either in words 
or graphically as seems most appropriate to him. She also 
asked for a definition of the role the teacher would be 
expected to play in preparing material and operating a 
system. 

Mr. Broderick developed this line of discussioh and said that 
the acceptance of this mode of learning by the teacher was 
vital to its fruitful development in the English educational 
system. It was also vital that the real contributions that 
teachers are willing and able to make should be given fair 
consideration. Many of them had a wealth of experience and 
were conscious of practical day-to-day problems and tech- 
niques which could play a vital part in the development of a 
CAI programme (c.f. the designer’s concept of the user men- 
tioned by Dr. Adams). 

He said that the teaching profession was a conservative one 
and had been known to take a hard line against seemingly 
logical innovations for reasons which had their roots in lack 
of trust and insecurity. He felt that CAI could easily be 
presented as a threat to the whole status and livelihood of 
the teacher instead of as an additional educational resource 
to the teacher’s armoury. He felt that the inclusion in 
teacher training courses of a detailed study of the ideas 
underlying the structuring of learning material could have a 
beneficial effect on the students both as regards their 
potential as teachers and as regards the readiness with which 
they would accept computer based learning systems later. 

The discussion now turned towards hardware and Professor 
Hansen said that the operational educational requirements 
were going to be difficult to define. Generally he advocated 
an economic approach with a limit set to cost within which 
the customer’s requirements should be optimised. As regards 
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the debate about large systems or small his preference was 
for the development of small ones. He was optimistic about 
what could be done with terminals and predicted the 
development of cheap graphic displays. Professor Hansen 
pointed out that computers have not hitherto been designed 
for CAI work, their logic was cumbersome where, for 
example, character manipulation was concerned. Only one 
peripheral was served and more overlap was needed. Disc 
files were organised for random access whereas the type of 
access structure required was reasonably well known. 

Mr. Hodgson said that conversational interaction in CAI was 
somewhat counterfeit in that the student freely constructed 
a response but key word analysis was a simplification of a 
freely constructed tutorial diagnosis — as Dr. Adams pointed 
out, the students were very perceptive of the real opera- 
tional constraints imposed on them by the terminal. The 
Centre for Structural Communication group together with 
GEC Hirst Research Centre had developed the “Syste- 
master” terminal concept. They had a simulation of a com- 
puter terminal which involved a combinatorial matrix res- 
ponse system which used overlays of subject statements, a 
code index output and a set of operating buttons. This gave 
“on-line” dialogue with a looseleaf book format. The 
approach made, by its form, the conventions of “dialogue” 
explicit for the student. The game he was playing was not 
made to look like something else. Since complex subject 
data could be symbolised by one bit of computer data, the 
system would enable several hundred terminals to be con- 
trolled by a medium sized computer. Normally available 
curriculum materials could be integrated into the system. 

Mr. Duerden said that reference had been made, in both Dr. 
Adams’ and Dr. Zinn’s papers, to the importance of the ter- 
minal problem, and both had mentioned the plasma panel 
display. He wished to emphasise the importance of an 
integrated design approach, bringing in engineers and even 
production engineers at a very early stage where engineering 
economics are recognised as crucial to system design. An 
engineering evaluation of the plasma panel initiated at Mar- 



coni some fifteen months ago had indicated that the panel 
itself was a well-engineered device basically suitable for 
quantity production, and with interesting design features 
likely to give it a very long in-service life. However, the 
penalty for these features was that there was a voltage in- 
compatibility between the panel and normal computer logic 
circuits, which seemed to preclude the possibility of an 
economic design. Since he regarded the plasma panel as a 
device suitable for filling the gap until the advent of fully 
solid state matrix displays and taking into account rapid 
advances in the latter area, he felt that effort to solve this 
compatibility problem was unwarranted. 

He mentioned other interesting display terminal develop- 
ments. The Direct Vision Storage Tube showed promise — 
selective erasure was now possible, and the problem of 
running cost had been overcome. His estimate of running 
cost of the early MIT terminals was about $ 1/hour for tube 
life. Long life tubes were now available from UK sources, 
and this particular cost factor was now vanishingly small. 
However, cost might still prevent the DVST terminals get- 
ting down into the price bracket required for extensive CAI 
use. 

Another interesting trend was the advent of cheap local 
storage as a result of MOS technology and medium scale 
integration. This led to fundamentally cheaper production 
processes, giving rather slower logic circuits; but speed was 
not important at the man/machine interface. 

The possibility of a major decline in cost was in the price of 
bandwidth. Hie thought that eventually a coaxial cable con- 
nection would be as standard as a gas or electricity supply; 
and this led to the concept of cheap terminals based on T/V 
technique. Indeed the terminal might well be a T/V display, 
and the only cost to be considered would be that of the add- 
on interactive facility. There was much to be said for inte- 
grating the approach as between the requirements of educa- 
tional technology and the general developments in the 
communications industry. 
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INTRODUCTION 

Computer-Assisted Instruction (CAI) is now more than a 
decade old. Having moved from a conceptual ideal, CAI has 
both proven its operational feasibility and revealed all of the 
complexities of the educational world. Moreover, it has con- 
fronted the educator with the diverse requirements of a 
technological approach to instruction. Thus, the field of CAI 
has by necessity addressed itself both to models of the learn- 
ing process as well as to issues dealing with efficient tech- 
niques for curriculum development within the requirements 
of available computer technology. 

For the purposes of this paper, a brief introduction to a 
learning model for adaptive instruction will be presented in 
order to clarify the difference between the instructional 
process and the curriculum development process. In turn, 
more specific remarks will be made in regards to instruction- 
al strategies as these form the primary intersection between 
these two theoretical and empirical domains. And finally, 
the major portion of the paper will describe our experiences 
at Florida State University in developing an autonomous 
multi-media computer-based collegiate physics course. Iri 
this final section, a “systems approach” model to CAI cur- 
riculum development will be presented. In order effectively 
to evolve and utilize the systems model in the development 
of the FSU physics course, ten significant professional roles 
in multi-media CAI curriculum developments will be des- 
cribed. The paper will then conclude with a set of summary 
propositions concerning the area of curriculum development 
within the CAI world. 

INSTRUCTIONAL LEARNING MODEL 

For the purposes of clarification, it is important when devel- 
oping CAI materials to have some conception about the 
learning process being utilized by the student. Moreover, as 
has been redundantly asserted, CAI is justified by its 
individualization of the learning process. In conceiving of 
the individualization of the learning process, most educators 
have tended to define the process as one of supplying appro- 
priate instruction to satisfy the student’s needs. This asser- 
tion is ambiguous at best. For example, are the needs to be 
defined in terms of the student’s frame of reference, especi- 
ally in terms of his wants? Or is it to be defined in terms of 
some benevolent power who controls what the student 
should have? The concept of needs is an integral behavioral 
construct evolving out of research within human motivation. 
The problem of its definition can be witnessed within the 
literature of human motivation and personalities process. As 
an alternative theoretical approach, one can specify a simple 
input/output model for the student and utilize this model to 
consider some of the preliminary factors in CAI curriculum 
development. 

Turning now to this simplified model, individualization of 



learning can be thought of as a process by which the student 
maximizes his informational input, mental processing, 
memory storage, and response output. In psychological 
terms, this conception of learning behaviors specifies the 
stimulus array, the cognitive processes, and the response 
requirements. Breaking the behavioral processes of learning 
into these three components will bring into focus some of 
the potential CAI curriculum development factors to be 
discussed in a later section of this paper. 

In regards to stimulus input, investigators such as Briggs and 
Gagn5 assert that greater learning gains can be achieved by 
appropriate assignment of instructional media. Matching 
appropriate films, audio lectures, or printed material to 
individual characteristics should, it is claimed, lead to better 
learning results. Current work in the area of Individualized 
Prescribed Instruction (IPI) and our own experience with 
CAI indicate that the assignment of appropriate media with- 
in CAI is a highly complex problem. For example, there are 
research findings indicating that cathode-ray tube presenta- 
tions to low ability students may in fact deter the learning 
process, or that audio lectures in some cases prove superior 
to film p re se n tatio n v -eveiL though there is an obvious rever- 
sal in terms of information characteristics of the two media. 
As a consideration within a CAI research project, it is there- 
fore important to prepare a design which allows for an 
assessment of the various media being utilized. As a feature 
in the adaptive nature of a CAI curriculum, alternative 
media approaches will ultimately provide useful insight as 
to range and optimality of each in a given curriculum. 

In regards to internal processes, the middle component in 
this simplistic model, the manipulation of the level of dif- 
ficulty of the learning materials has proven to be a powerful 
variable. In research at Stanford University, as well as at a 
host of other CAI Centers in the United States, it is clear 
that optimal matching of the level of difficulty of the learn- 
ing materials to the student’s performance level leads to im- 
proved processing as well as enhanced long term retention. 
As an example, a recent study in our laboratory indicated 
that the use of concurrent memory retention indices provide 
enhanced learning in comparison with more general indivi- 
dual difference variables such as an IQ score. While a large 
array of alternative psychological models can be proposed 
for this internal processing by the student, it is important to 
consider within CAI curriculum development such simplistic 
factors as the scaled information load as evidenced by a 
readibility indices, the complexity and sequential structure 
of solution algorithms, and finally the fostering of long-term 
retention. 

Turning now to the response side, the third component in 
the model, it would seem that most CAI curriculum develop- 
ment projects have constrained themselves by the availability 
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of computer/ terminal equipment. Encouragingly, though, 
most students indicated a quick adaptation to the response 
requirements of the student/computer interface with little 
or no detrimental effects from one alternative device as 
opposed to another. To be more specific, very young 
students have clearly demonstrated the ability to master the 
typewriter keyboard, or no evidence exists as to the superi- 
ority of an electronic blackboard as opposed to a more in- 
expensive keyboard device. As a wider array of curriculum 
materials are developed, it may become clear that more 
appropriate matching of response characteristics of student/ 
computer interface may foster more optimal learning. CAI 
curriculum projects may desire to be more exploratory in 
the area of alternative response devices. 

While acknowledging that this input/output model for 
individualized learning is extremely simple, it provides a CAI 
curriculum project with the essential considerations in think- 
ing through each specific stage in the curriculum develop- 
ment process. A failure to consider the student and his re- 
lated behavioral processes has been one of the major flaws 
in many of the CAI developments to date in the United 
States. It is also important to indicate that there has been 
little experimental investigation in regard to appropriate 
matching of learning, computer, and curriculum character- 
istics. Until this void is eliminated, major CAI curriculum 
developments will be limited in regard to their implementa- 
tion and implications. 

INSTRUCTIONAL STRATEGIES 

The major intersection of a model of the learning process 
with that of a CAI development model comes under the 
rubric of instructional strategies. This term was first referred 
to by Stolurow in terms of the logic flow of the instruction, 
that is, the branching structure utilized within the context 
of correcting error responses or applying remedial pro- 
cedures. As a contrasting conceptual frame of reference 
Smallwood proposed a quantitative model by which to de- 
fine instructional strategies that lead to optimal solutions 
for the learning outcome. From my point of view an instruc- 
tional strategy is one that allows for selection from the 
alternative plans of instruction the one that hopefully will 
lead to an optimal performance level. These instructional 
plans involve the characteristic of the learner, the structure 
of the curriculum material being developed, the behavioral 
processes' being utilized by the student, as well as the stu- 
dent’s coping behavior that results in maximizing his re- 
wards and minimizing his efforts. Thus the student, from my 
point of view, will always try to maximize his rewards and 
minimize his efforts in terms of either playing an “interest- 
ing game” or contending with the problems posed by an 
educational system. 

The primary issue concerns who selects and controls the in- 
structional strategy. At one end of the continuums Stolurow, 
Smallwood, and Atkinson would suggest that we prescribe 
the optimal selection of learning events for the student. 

They claim that having once understood the student’s basic 
behavioral processes that we, as an outside decision-making 
mechanism, can best decide his prescription for instruction. 
At the other end of the continuum, Grubbs has suggested 
that a student, given his better self-awareness of all of his 
internal mental processes and immediate state of under- 



standing, can best select his own strategy for acquiring a set 
of complex concepts. For my part the process for the selec- 
tion of instructional strategies should be considered one of 
negotiation between the instructional system be this a 
teacher or a computer and the student. This negotiation 
should allow for more student initiative and self-selection 
given better desire performance, that is, the better the per- 
formance by the student the more we offer him self-selec- 
tion among the learning topics, alternative media, and 
criterion levels of performance. Recent work in the area of 
social learning contingency games indicate that allowing for 
student initiative leads x o at least these two results: (1) more 
student accomplishment of the desired performance defined 
in terms of behavioral objectives in less time, and (2) more 
motivation by the students to move towards the category of 
superior performance. Thus CAI curriculum projects must 
constantly consider the social learning contingencies if a 
successful overall instructional course is to be developed. 

The frame by frame issues typically discussed within pro- 
grammed instruction appear to be marginal in their impact 
on CAI learning. In essence I am recommending that a wider 
and richer approach to instructional strategies with more 
student involvement will provide better payoffs in learning. 
We turn now to the specific issues in CAI curriculum devel- 
opment. 

SYSTEMS MODEL FOR CAI CURRICULUM 
DEVELOPMENT 

The systems approach has evolved as a set of ideal analysis 
and implementation procedures that can be followed in 
order to develop effective learning materials which in turn 
maximize the conceptual development of the students. The 
essential features of the system model are schematically 
presented in Figure 1. The first step in the process is the 
exploration and description of the instructional problems 
plus associated context constraints of the instructional 
setting. Concurrently, a task analysis of the conceptual re- 
quirements, as well as the behavioral processes, should be 
performed, A thorough assessment of the entry skills and 
prior knowledge of the student population for which the 
course is intended is also required. These sub-analyses then 
culminate in the course behavioral objectives which form a 
description of the criterion performances which are desired 
as outcomes for the student. In turn, the behavioral objec- 
tives are sequenced and structured into instructional strate- 
gies for given segments within the course. As a consequence, 
appropriate selection of media and instructional contexts 
provides the implementation prior to the first field test. The 
empirical results obtained in the field test provide the basis 
for evaluation and subsequent revision cycles. 



While this is an overly simplified representation of the pro- 
cess, each of the system’s components will be described in 
more detail below. The adaptation and utilization of this 
model by the FSU project staff will be emphasized. 

1 . Problem Identification 

In the process of identifying the existing instructional prob- 
lems within the physics course, it was found useful to 
employ a number of techniques by which to reveal specific 
problems upon which the CAI approach could focus. If con- 
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ceptual learning problems can be identified in terms of 
behavioral phenomena such as prior test scores or responses 
on homework assignments, etc., a CAI project will be much 
further ahead in its formularization of appropriate behavior- 
al objectives. 

Four techniques were utilized to identify problem areas 
within the physics course. First, a thorough literature search 
of the physics education area provided information about 
the needs of students for prerequisite quantitative abilities, 
for high order abstracting and concept formation abilities, 
and for sophisticated problem-solving skills. In the last 
analysis, it was apparent that one learns physics to the de- 
gree that one can solve physics problems. The primary be- 
havioral focus on problem solving for physics courses should 
not be minimized. 

The second technique involved a number of conferences 
between members of the FSU physics faculty and the pro- 
ject staff in order to gain case study information about 
learning problems revealed during class discussion periods as 
well as faculty office hours. These conferences pointed up 
the need for good conceptual development and associated 
problem-solving skills plus the deficiency of student motiva- 
tion for certain aspects of the course. These motivational 
factors seem to determine class attendance, work effort, and 
general intellectual commitment. 

In terms of the third technique, all of the prior test results 
over the previous three-year period provided a clear indica- 
tion that the later portions of the course, namely electro- 
magnetic phenomena and atomic physrcs, provided the 
greatest difficulty for the students in terms of items failed 
on final examinations. 



The fourth technique for identifying difficult concept topics 
leads to a set of CAI physics problems which were presented 
on four different occasions to samples of students enrolled 
in the conventional physics course. The performance of the 
students on these CAI instructional problems provided per- 
formance data upon which all future comparisons for re- 
vision and improvement purposes were based. The availabil- 
ity of baseline data is an extremely useful technique and 
should not be minimized. 

All of these efforts clearly indicated that throughout the 
physics course there were specific learning or conceptual 
problems that influenced the overall performance into a 
gradual decline as the students proceeded through the 
course. 

2. Task Analysis 

A task analysis of the curriculum concepts to be taught to 
the students provided an overall structure of the course con- 
tent in a manner that delineates the relationship among 
topics in both sequential and hierarchical fashion. In terms 
of introductory physics, the integrating conception of 
particle and wave phenomena provides a recurrent and in- 
creasingly complex set of theoretical propositions as the 
student moves through the topics on measurement, optics, 
mechanics, electromagnetism, and modern physics. This 
relatively stable conceptual structure has evolved over a long 
period of time and is easily inferred from a review of exist- 
ing textbooks. 

For the purposes of the project, the task analysis of the con- 
tent was performed in two ways. First, a video recording 
was made of the twenty-nine conventional classroom lec- 
tures and demonstrations. These video tapes provided an 



opportunity to study both the detailed presentations of 
concepts, but more importantly to identify the language 
and representatives utilized in the conventional setting. 
Parenthetically, it is highly recommended that video record- 
ings of a professor who is highly successful in conventional 
teaching provides many important insights into the peda- 
gogical techniques and language appropriate for instruction 
in a given course area. Moreover, the video recordings allow 
one to identify the characteristics of concept presentations 
which will be of value when consideration is given to media 
assignment. And, lastly, it provides an invaluable tool by 
which the professor can compare and reconsider the se- 
quencing of portions of the course. 

As a second task analysis technique, four currently popular 
physics textbooks were analyzed. Interestingly enough, the 
topic sequence in all of these textbooks was exactly equiva- 
lent; that is, the authors employed the concepts of particle 
and wave phenomena in order to integrate the topics within 
the introductory physics course. As an additional benefit, 
the analysis of the homework problems required at the end 
of each chapter indicated many of the behavioral require- 
ments currently considered important in introductory 
physics. 

3. Entry Behaviors 

An empirical assessment of the skills and performance level 
of the student population as they enter a course is an abso- 
lute prerequisite for the preparation of optimal learning 
materials. These performance levels are commonly referred to 
as entry behaviors. Entry behaviors represent a characteriza- 
tion of the heterogeneity of both cognitive and affective 
processes and prior knowledge levels on the part of the 
students. Obviously, as gaps or deficiencies are revealed, these 
impinge directly on the conceptual attainment as represented 
in the task analysis. In essence, entry behavior should indicate 
both the aptitudes and abilities of the students at the 
beginning of the course and the appropriate entry points into 
the conceptual flow identified within the tasks analysis of 
the course. 

Th: 1 entry behaviors of the FSU students were assessed in 
terms of scores on the Florida Collegiate Entrance Examina- 
tion, performance on midterm and final examinations in the 
conventional physics course, and most importantly the per- 
formance on the CAI problem sets. These CAI problem sets 
were a fair representation of each of the sub-concepts pre- 
sented in the conventional setting. The students typically 
came to the CAI Center prior to each examination for one 
to two hours of instructional interaction. Each CAI item 
poses a physics problem; if the student could not answer it, 
help was provided until a successful answer was emitted. 

The preparation of this type of CAI complementary problem 
set is highly recommended in order to identify specifically 
the performance level of students both prior to and during a 
conventional course preparation. 

Problem sets have great merit in that they save a great deal 
of time and energy in terms of preparing desired remedial 
materials and delimiting professors’ and authors’ intuitions 
about potential learning problems. The area of CAI curricu- 
lum development has been fraught with extensive remedial 
material preparation which is rarely used by any of the 



targeted students. It was discovered that utilizing the CAI 
homework problem results saved considerable time and 
focused the preparation of learning materials specifically on 
difficulties demonstrated by concurrently enrolled students. 
Thus empirical techniques provide an efficient approach to 
specifying student entry behaviors. 

4. Behavioral Objectives 

Information from the course analysis, task analysis, and 
entry performance levels was utilized in formulating the 
behavioral objectives of the CAI physics course. Since a 
direct comparison with the conventional course was desired, 
the concepts and related behavioral objectives were arbitrar- 
ily divided into twenty-nine segments referred to as lessons. 
These closely parallel the presentations in the conventional 
lecture-demonstration course. Tile behavioral objectives 
were treated as hypothesized propositions which could be 
and ought to be achieved by the students given an effective 
instructional treatment. 

For each lesson the behavioral objectives were broken down 
in terms of prerequisite skills and concepts plus the behavior- 
al objectives for that given instructional segment. It was 
observed in the process of stating the behavioral objectives 
that the availability of prior test items as well as the video 
recordings of the conventional class presentations proved an 
invaluable data source from which to formulate precise per- 
formance related statements. These precise behavioral ob- 
jectives assist one in the next step, namely, forming instruc- 
tional strategies. 

S. Instructional Strategies 

Since the conceptual structure of the collegiate physics 
course did not pose major sequencing problems because of 
the constancies within existing text books as well as the 
equivalent structure or reverification from the CAI task 
analysis, the instructional strategies focused on the convey- 
ance of appropriate learning expectancies to the students via 
various types of media presentations. Distinctive instruction- 
al strategies were utilized for each of the subsections of a 
lesson. 

First, each textbook reading assignment was followed by a 
detailed CAI quiz which had a specified criterion perform- 
ance level. If a student failed to meet criterion, he was given 
a remedial reading assignment and recycled through the quiz 
items. This strategy insured that the students’ comprehen- 
sion of the text was more than sufficient. In regard to the 
audio lectures, a set of typed notes apd diagrams were 
utilized in conjunction with the audio tapes. The concepts 
presented in the audio lectures again were evaluated in terms 
of CAI quizzes. For remedial purposes, students were re- 
quired to repeat the presentation if their performance did 
not meet criterion. For both the physics conceptual film 
presentations and the laboratory film loop presentations, 
there were related CAI quiz items. Again, students were 
directed to return to the presentation if their performance 
was not at or above the desired criterion level. 

In each of the lessons, the final assessment of the behavioral 
objectives was in terms of a CAI problem set. Students were 
provided detailed remediation within the structure of each 
of the problems. As a follow-up, a parallel form of the 
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physics problems was presented as review material prior to 
both the midterm and final examinations. These CAI review 
problems again assessed the long-term retention of the 
behavioral objectives for each of the lessons. 

In essence, the instructional strategies were created in order 
to relate hypothesized sets of psychological states through 
which the student would pass while completing various 
tasks in each of the physics lessons. In this regard, the 
students were provided a recognition of the learning 
expectancies to be covered within each of the sub-sections 
of the physics course. This was accomplished via explicit 
directions plus criterion quizzes at the end of each sub- 
section. These psychological expectancies provided involve- 
ment and commitment on the part of the student to obtain 
the desired behavioral objectives. Without this psychological 
commitment, there would be a low probability that the CAI 
instruction would produce the desired optimal learning out- 
comes. 

Having gained the student’s involvement, the new informa- 
tion of each lesson must be sequenced in light of the prior 
knowledge and problem-solving skills gained in prior lessons 
by the student. The algorithms of these problem-solving 
skills are clearly related to the specific sub-concepts of each 
topic in the physics course. For example, the solution of 
kinetic energy problems related back to considerations of 
the sub-concepts of force and matter. If a student had 
mastered the sub-component elements of each concept, then 
the more complex algorithms could be applied. 

As a last feature of the instructional strategy, an attempt 
was made to provide frequent conceptual closure and the 
self-realization by the student of having gained competency 
over each specific topic in the course. This psychological 
requirement for frequent closure is one of the most over- 
looked aspects involved in effective instructional strategies. 

6. Media Assignment 

As a related aspect of the development of the automated 
physics course, the process of assigning appropriate media 
for each concept is critical. Most of these decisions are 
typically based on relatively unexplored research con- 
ceptions. Obviously, the media utilized for a given presenta- 
tion has to be as contiguous or as similar to the response 
modality as possible. The physics course utilized a wide 
variety of multi-media modalities. Rather than restricting 
the presentation only to the CAI-CRT terminal device, the 
most appropriate match between the media and the informa- 
tion features of the concepts was attempted. This use of 
multi-media within the physics course offered an oppor - 
tunity to analyze the learning impact of these media types. 

The following guidelines were used for media selection. 

First., when attempting to facilitate acquisition of con- 
ceptual material, the use of multiple sensory channel inputs 
was maximized. For example, in presenting a complex 
demonstration of physical phenomena like kinetic energy, 
either PSSC films or film loops were used in order to maxi- 
mize the richness of the sensory characteristics. Second, 
when allowing for both acquisition and intellectual problem 
solving, the information source was focused within restricted 
sensory channels. For example, many problem-solving routines 



were illustrated within the audio lecture through the use of 
accompanying graphic presentations. Third, when attempt- 
ing to build problem-solving skills for long-term retention, 
the use of feedback and correction via CAI was maximized. 
The interactive feature of the CAI system was utilized in 
order to individualize the feedback, the correction, and to 
insure sufficient practice. Fourth, when faced with evalua- 
tive decision-making, especially in determining successful 
attainment of the behavioral objectives, the real-time 
student history feature of the CAI system was utilized in 
order to scan over a number of learning tasks in determining 
an appropriate decision about criterion performance. And 
lastly, the logistics of the instruction from the student’s 
point of view in moving from one media device to another 
was considered. While interruptions may break the 
monotony of the instructional process, it has been found 
that interruptions within learning processes can interfere 
with acquisition and retention. Thus, an attempt was made 
to match appropriate media in order to have a smooth flow 
through a given lesson. 

7. Field Tests 

In conducting the field tests and subsequent revisions, the 
following factors seemed important based upon our 
experiences. First, appropriate selection of students who 
vary according to aptitude, prior knowledge, and other 
psychological characteristics is difficult to obtain but 
important. The forming of special sub-groups to assess their 
reaction to the materials formed the substance of all future 
revisions in the CAI physics course. Secondly, the import- 
ance of looking at learning frame statistics as well as overall 
course performance became quite contingent upon our 
ability to process and analyze the CAI data encoded within 
the computer system. As will be explained in a subsequent 
section, a computer data analysis and management system 
was developed in order to perform these analyses. Various 
reports proved invaluable to the course authors in the 
revision process and should be considered an essential part 
of any computer approach to instruction. Third, good inter- 
view techniques should be employed constantly, not just at 
the end of the course, blit throughout the instructional 
process. Informal comments from students can be treated as 
hypotheses which need to be checked out as to their 
validity and potential implications for course change. The 
informal comments from students concerning scheduling 
and the reliability of various media devices indirectly formed 
the basis upon which certain equipment and scheduling 
changes were made in the CAI physics course. Lastly, a pool 
of experienced personnel with clear understanding of their 
functions is required when one is pursuing development 
work in computer approaches to instruction. For example, 
the primary function performed by the student proctors 
was one of assistance to the students, but more importantly 
they served as input sources by which important informa- 
tion was gained both through direct observation of and 
interactions with the students. 

8. Field Study and Project Development Schedule 
Table 1 (see next page) presents a brief quarter by quarter 
description of the primary project activities. It can be 
observed that most of the first year was devoted to develop- 
ing the course. The first field study was conducted in the 
fall of 1967. The second field study, the most complete of 
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Table 1 



Developmental Schedule for the Project 





First Year. C 1966 — 7) 


Second Year, (1967—8) 


Third Year, (1968-9) 


Fall 


Project Initiation Staffing, 
CAI Problem Exercises, 
Course Analysis 


First Course Field Test, 
CAI Problem Exercises 


FLEX 
Field Test 


Winter 


Video Recording, (1967), 
CAI Problem Exercises, 
Task Analyses 
Entry Behaviors 


Data Analysis, (1968) 
Course Revision 


Data Analysis, (1969) 
Final Report, 

Project Ended 


Spring 


CAI Problem Exercises, 
Behavioral Objectives, 
Film Preparation, 
Course Authoring 


Second Course Field Test 
CAI Problem Exercises 




Summer 


CAI Coding, 

Audio Loop Preprations, 
Graphics Preparation, 
CAI Problem Exercises 


Data Analysis, 
Course Revision 

_ 





the experimental versions, was presented in 1968. The final 
field study that focused on individual difference outcomes 
was completed by December, 1968. This project schedule 
offers at least one example of the time requirements to 
develop a collegiate CAT course. 

MANAGEMENT TECHNIQUES 

The primary task in the management of the project consist- 
ed of evolving and redefining the functional roles for staff 
personnel. As new needs and related functions were 
identified, a staff member assumed the responsibility and in 
essence created the role. The primary mechanism for plan- 
ning and co-ordinating was a weekly staff meeting. While 
more formal project planning techniques like PERT might 
have improved the project’s development, the unknown 
nature of the CAI course development process resulted in 
the use of more informal planning and communication 
techniques. However, the use of the Systems Approach as a 
model for CAI curriculum development guided our efforts. 

Any CAI project utilizing a rich array of technological 
equipment requires a complex functional organization that 
differentiates roles and related competencies. This section of 
the paper describes the various roles which evolved within 
the physics project. 

1 . Content Scholars 

Foremost within a CAI project is the requirement for 
excellent subject matter scholars who have complete 



command of the concepts to be taught. The project was 
fortunate to have the involvement and professional commit- 
ment of four professors from the Florida State Department 
of Physics. While the project did not create a major new 
sequencing of the concepts of physics, each of these men 
provided excellent insights within the following phased 
steps. 

First and foremost, these professors devoted innumerable 
hours to the preparation of a detailed conceptual outline of 
the course. In addition, they allowed us to video-record 
their classroom presentations over two successive quarters. 
These video recordings were used to study the language and 
demonstrations utilized in these lecture presentations. As 
various segments within the CAI course were developed, 
each of the four professors provided valuable contributions 
in terms of critiquing and editing the course materials. 

Since these materials were automated, these professors went 
through them in a student mode in order to detect any 
misconceptions or inaccuracies. In addition, the professors 
provided invaluable service in the continual preparation of 
new sets of midterm and final examination questions as well 
as the homework assignments for the physics class. It should 
be noted that a common set of examinations and homework 
problems was utilized in comparisons between the conven- 
tional and CAI versions of the course. 

Informally, these physics professors also contributed to the 



development of the field studies by lending professional 
support to the process of gaining permission to teach the 
CAI version for full university credit. While this may seem 
like a minor point, one should not minimize the time and 
energy required to gain permission to offer credit for an 
experimental instructional course. Typically, the university 
administrators wish the assurance that the “new” course 
will be equivalent to or better than the existing course. 

Prior to the first field study, as many arguments as possible 
for accrediting the course were assembled with the know- 
ledge that the empirical outcomes might in fact refute some 
qf the claims. 

2. Behavioral Scientist 

An equivalently important talent is represented by the 
behavioral scientists who provided insights into the overall- 
creation and implementation of the systems approach. 

Being “behavioral methodologists,” the behavioral scientists 
provided reasonable criteria for the behavioral consequences 
of the instruction. They also analyzed the issues dealing 
with the topics of entry behaviors, task analysis, behavioral 
objectives, and instructional strategies. Concurrently, the 
behavioral scientists contributed the major structure of the 
research design as well as specific hypotheses which are 
reported within the field studies. Since they had prior 
experience with experimental data analysis, the responsi- 
bility for analyzing the instructional outcomes and 
interpreting them also was assumed by the behavioral 
scientists. Perhaps most importantly the behavioral scientists 
provided the managerial leadership and the training of other 
personnel within the project in order to achieve the project 
goals. 

3. Physics Writers 

Since the talents of both the professional physicist and 
behavioral scientists are in exceedingly short supply the 
project recruited a full-time physics writer. After being 
trained in the nature of CAI and the desired instructional 
strategies, plus viewing the video tapes for appropriate 
language, the full-time writer, as well as three part-time 
physics graduate students, proceeded ahead with the 
detailed writing of the instructional materials. Thorough 
command of the physics content and an understanding of 
the overall systems approach and computer capabilities were 
required. The majority of the writing was performed by 
these authors. It can be recommended that such full-time 
writers form an essential ingredient in a reasonably large 
CAI developmental project. 

4 . CAI Coders 

After the instructional material has been edited, a CAI coder 
entered it into the FSU-IBM 1500 CAI system. The CAI 
coder had a thorough understanding of the Coursewriter II 
language, the uses of switches and counters for real-time 
data analysis, and the role of macros which provide a 
method for more quickly encoding curriculum materials. 

The CAI coders, who are excellent typists, typically per- 
formed both the entry and copy editing functions; that is, 
many minor mistakes were picked up by these coders and 
referred back to the physics writers and the physicists. This 
type of informal editing can be exceedingly important with- 
in the implementation phase of CAI. 



5. Media Specialists 

In terms of the physics project, part-time media specialists 
were employed who helped in the preparation of the con- 
cept films as.well as the audio tapes. Since a random access 
audio system was available for this project, instruction in 
the preparation of tapes was required. While no special or 
unique functions evolved for these media specialists, they 
did prepare all of the final version of the curriculum. 

6. Computer Operators 

As the physics course was being encoded by the CAI coders, 
a computer operator had to be available for supervision and 
normal back-up operations on the computer. The primary 
contribution of the computer operator was in terms of solv- 
ing linkage failures within the CAI courses. These linkage 
failures are computer errors which drop required indices 
that correctly link up various branched parts of a CAI 
course. In addition, the computer operators kept a very 
extensive set of records as to the nature of the CAI opera- 
tion and scheduled work loads, so that appropriate materials 
were available for all students. 

7. Computer Systems Programmers 

In the process of developing the course, it was necessary to 
employ a computer systems programmer who developed the 
FSU Data Analysis and Management System. In addition to 
designing overall systems for CAI operations (e.g., more 
effective ways of encoding materials for data analysis, or 
more effective reports for authors and investigators), the 
computer systems programmer focused on the logistics of 
the total computing system. Resolving certain logistics 
problems, such as the requirement for extensive course list- 
ing, etc., has been very- important within the CAI context 
in order to insure prompt processing of all requests. More- 
over, the systems programmer has developed special Course- 
writer functions that allow an author to gain the kinds of 
information and branching flow desired within the instruc- 
tional sequence. Thus, the overall computer system was 
vastly improved by the computer systems programmer. 

8. Data Analysis Programmer 

Repeated data analyses, especially in terms of item frames, 
were required as a critical part of the project. This function 
typically involved taking data from the CAI data manage- 
ment system and processing it on any of the computers on 
the FSU campus. While many of these statistical programs 
such as items analysis and linear regression were available, 
the preparation of new input/output statements were a 
special requirement for the project. 

9. CAI Proctors 

As mentioned in the description of the field study, a proctor 
is necessary to supervise the actual mechanics of CAI in- 
struction. The primary activity in the physics project was 
assisting students in preparing various media devices for 
actual utilization. Proctors had competencies in physics so 
that they could assist students with conceptual problems. 
However, these problem-solving requests were so infrequent 
as to be almost non-occurring. In addition, the proctors 
kept extensive observational notes and performed interviews 
which provided a great deal of information related to the 
student’s adaptation to the multi-media CAI physics course. 



10. Graduate Students 

Within any large CAI curriculum development project there 
should be an array of graduate students who can provide at 
least two significant contributions. First, the graduate 
students represent excellent back-up personnel and superior 
problem-solvers. The physics project was inundated with a 
multitude of small problems and our graduate students 
learned a great deal by resolving them. More importantly, 
though, the graduate students continually raised questions 
about the overall Systems approach and generated small 
research experiments related to major questions revolving 
around instructional strategy and media selection. This 
small-scale experimental research performed on other con- 
tent topics provided important information during the 
formative stage of this project. Thus, it is felt that the 
support and active involvement of graduate students is an 
important ingredient in the overall mix of functional roles 
in a complex CAI project. 

DATA ANALYSIS AND DATA MANAGEMENT 

As a result of the need for data analysis in the CAI physics 
project, a general file structure system was developed that 
allowed for the organization of each student’s behavioral 
responses into a clearly identifiable file array. This general 
file structure is an exceedingly important feature in data 
analysis for a number of reasons. First, authors tend to be 
primarily interested in item or frame statistics. The file 
structure must be manipulatable so that item and frame 
statistics can be printed out in a number of ways in order to 
characterize performance and allow for easy inference mak- 
ing in the revision process. As a corollary, the quick avail- 
ability of this information for the authors is exceedingly 
important. Secondly, the file structure must be amenable 
to comparative analysis for various portions of the course, 
or various media presentations. These comparative analyses 
permitted the project team to decide whether certain 
hypotheses were in fact valid and worthy of further pursuit. 

In terms of more sophisticated analyses, a number of 
factorial and linear regression techniques were utilized in 
order to obtain both with and across group comparisons. 

The data file structure was organized in a matrix fashion in 
order to generate variance and covariance matrices which 
could be utilized within these regression models. These 
linear regression analysis techniques are extremely useful in 
gaining insights into the identification of variables which are 
important in terms of positively influencing the perform- 
ance levels resulting from the instruction. 

One of the great potentials of CAI data is the sequential 
tagging of each student’s response. The sequential analysis 
of responses has proven to be of considerable difficulty and 
the FSU CAI Center is still developing programs to allow 
for more adequate analysis of sequential responses as well as 
latencies. Ultimately, it is hoped that these analyses will 
eventuate into quantitative models that characterize the 
learning process. Unfortunately, the complexities of the 
analysis have prevented this avenue from being pushed much 
beyond the linear regression models. Thus, it is felt that the 



investment in and development of the Data Analysis and 
Management System was an important ingredient for the 
successful completion of this project. 

SUMMARY 

This paper has primarily described the CAI curriculum 
activities of the FSU Physics Project. Unfortunately, there 
are few empirical reports from other CAI curriculum pro- 
jects in the United States that describe their developmental 
procedures. Informal discussion and communications with 
these other CAI projects indicate close similiarities to our 
efforts at FSU. In light of these similiarities, the following 
eight factors seem critically important in determining the 
rate of development and success of a computer-based 
curriculum project: 

1 The use of the systems approach and the clarity of the 
behavioral objectives derived for the CAI curriculum will 
determine the rate with which a project will be developed. 

2 The variety and frequency with which varying response 
modalities such as speech, light pen. keyboard, etc., are re- 
quired in a course can affect the rate at which a CAI 
curriculum can be implemented. 

3 Terminal criterion performance levels for the CAI course 
will determine both the instructional sequence as well as the 
complexity of the instructional strategy. In turn, the com- 
plexity of the instructional strategy will determine the 
developmental rate of the project. 

4 The variety of multi-media utilized in the CAI course will 
determine the implementation rate and the logistic ease of 
the instructional process. 

5 The number of revision cycles required to develop an 
“optimal version” of a CAI course remains an unanswered 
question. However, the use of CAI problem sets to deter- 
mine baseline performance and video recordings of excellent 
instruction in a conventional setting allowed for restricting 
the number of revision cycles. 

6 The degree of sophistication of the CAI operating system 
is highly critical in determining the rate of development. 

The availability of an efficient coding language with macro 
techniques plus an operative computer data analysis and 
management system is highly essential for a favorable rate 
of development. 

7 The number of experimental versions of the CAI course 
will determine the rate with which the project successfully 
reaches closure. However, investigation of experimental 
issues is necessary for the full evaluation and validation of 
the curriculum. 

8 Since it is recognised that CAI curriculum development 
is a highly complex process, the use of multiple role 
differentiation techniques and specific functional assign- 
ments for staff members leads to more effective and 
efficient rates of development. 



DISCUSSION 

The session divided into two distinct sections; the 
first dealt with points arising from Hansen's paper, 
the second consisted of prepared statements describing 
developments in Louvain, Glasgow, the Centre for 
Structural Communication and H.M.S. Collingwood. 

Prof* ssor Hansen spoke briefly enlarging on the 
thi i .ing behind his paper and in particular spoke of the 
value of having a generalised learning model. All the 
instructors had some conception of student processes 
but the model adopted gave some common framework and 
was particularly useful in explaining the work to anyone 
not directly involved. He outlined three possible 
strategies for task analysis: 

1. interviews with successful students. 

2. building up modular flow diagram of course. 

3. starting with desired objectives and then 
moving backwards until entry points were 
reached. 

He also emphasised the importance of getting the system 
in operation quickly, then revising by using data 
collected through the system, although he warned against 
collecting too much data at first. 

In answer to questions he replied that the course 
covered ground that would usually require 30 hours of 
conventional instruction (3 hours a week for 10 v/eeks) 
but that the average time saving on this was of the order 
10/15%. In addition there was a further 6 hours of 
revision material. Full details of the entire project 
had been published and was available, in 3 volumes, from 
Florida State University. There was the possibility of 
extending the material to other universities and there 
had been four formal requests for the entire course and 
altogether about twenty others were interested in some 
of the problem papers. 

Mr. D’Arcy then thanked Professor Hansen for his paper 
and pointed out that most CAI projects in the States 
were built up piece-meal and that one of the objects of 
this seminar was to enable us to benefit from their 
experience. He thought that the paper gave us, in effect, 
a very concise and clear do-it-yourself CAI kit, covering 
the learning model, the systems model, management 
techniques and data processing. He wondered if we could 
start by looking at one of the central themes of the work 
which was the interaction of the learning model with the 
systems model in the area of instructional strategies 
and in particular at the primary issue of selection and 
control. 

Dr. Annett stated that this was an area in which he 
was particularly interested and thought that it was one 
in which the computer could do well something that could 
not be done satisfactorily by other means. He wondered 
if Hansen would enlarge on his work. 

Professor Hansen explained that this matter of 
student choice was one that had interested him consi- 
derably and that he was still engaged in further work. 



(Among choices allowed to the students had been 
additional exposure to work and complete control over 
review material). He indicated that this work, and the 
work on the physics course , gave a clear indication of 
the gain to be made from allowing student choice to take 
place. These results were confirmed by work done in the 
University of Texas. He also stated that they were 
exploring the possibilities of multi-person experiments 
with two, three or four students using one terminal. 

Professor Pask outlined the continuum that existed 
between the learning strategy that allowed the student to 
explore freely his learning environment and that which 
used some feedback to control him. In between the two 
extremes was a ‘conversational’ system that allowed the 
student a choice of strategy. 

If the machine thought it necessary to override the 
student it could do so but the student was told why and 
the general effect was to obtain a compromise between the 
strategies preferred by the student and those preferred 
by the machine (there are more details of this position 
in the paper presented by Pask). There was evidence to 
show that the very good students did benefit in the free 
environment and that there was an average gain in the 
restricted one with the good students being slowed up. 
However, in the conversational systems there had been a 
dramatic increase in learning. 

Mr. Duke then asked if the discussion could be 
brought round to the areas of attack; should one 
choose simple or complex areas if one expected a good 
return. In particular he asked Hansen why he had 
chosen physics. 

Professor Hansen replied that he thought the 
really important thing was the people involved and that 
he himself believed in finding bright exciting people 
who would then select their own area. In doing this he 
had a very broad view of what constituted CAI. In Florida, 
for example, there was a group using a computer to study 
poetry. They used a batch processing system and the 
response time was several hours. He maintained that 
this was still CAI and that the important ingredient 
was the people who wished to work in that area. 

Dr. Adams looked for criteria to justify using the 
computer and thought this could be that the work must 
contain something that the computer could add and that 
responses at the terminal must be relevant to the 
objectives of the course. It was also important that 
student responses could be processed. 

Mr. Hodgson raised the question of cost and in parti- 
cular the payment to Faculty members in the Florida 
project. He thought that reducing costs was a critical 
issue in CAI. 

Professor Hansen replied that it was difficult to 
apportion costs in a large developmental project and 
referred to the 3 volume report which detailed this. He 
suggested that anyone interested in this should contact 
the university for these reports. 



Mr. Hartley thought that the difficulties of imple- 
menting and using a CAI system were consistently under- 
estimated particularly on the educational side. He thought 
this might be because evolutionary processes in education 
favour the teacher so that through the years a..system of 
classroom organisation, book methods, set syllabuses and 
public examinations had been developed: Computers would 
favour different more individualised methods but the 
present system seems reasonably efficient — and it -is the 
one in which computers must find a place. This has led 
to a way of implementing computer-based learning in which 
experienced teachers were found, quizzed about objectives, 
educational content and structure, the responses which 
they expected from students and the decisions they make. 
These were stored within the computer and all instructions 
duly carried out. He believed that such methods may be 
useful and necessary at the start but there are 
disadvantages. The material is expensive to produce 
because all the work had to be done by highly trained 
personnel, and although courses will benefit and be more 
efficient than conventional methods the teaching 
strategies will have been of the ‘ad hoc’ variety. Many 
variables were undefined and uncontrolled so that it was 
difficult to see how it could be improved. Because of 
this expense the material is not likely to be tampered 
with and the project would acquire much inertia. In 
addition to this all difficult decision-making and 
production of material had to be anticipated so that the 
educational use of the computer was rather trivial; it 
merely stored and presented information. He advocated that 
within large scale projects there should be smaller 
projects that allowed the computer to take over more t . 
the teaching decisions, to help in generating material 
and adapting to the individual student. 

He thought it may be necessary to set such experimental 
projects in areas which present particular difficulties 
to the teacher and tackle those functions in which the 
size of classes made him less efficient. He then 
postulated three phases of teaching; initial teaching in 
which the learner is introduced to certain ideas, concepts 
or techniques, supporting practice for this, and finally, 
problems in which the student applies what he has learnt. 

It was unlikely that the computer would be acting to its 
full potential in teaching pure content as the physical 
constraints of the interfacing equipment often made it 
less congenial than conventional methods. However, the 
computer could have a useful role to play in practice 
exercises. The notion of practice implies rules so that 
if these and the elements used together with levels of 
difficulty are specified then the computer could generate 
material, adapt to individual performance and provide 
close monitoring and control. Teaching in this way was 
economic, and typically, quickly produced learning gains 
over conventional methods. At the same time the computer 
could be used as a calculator and relieve the student of 
monotonous low level tasks. 

A much more interesting use was in problem-solving 
and simulation techniques and at Leeds such work was 
being carried out in aspects of science, mathematics 
and clinical diagnosis. Teaching material was stored 



so that the student could refresh his memory but it was 
assumed that he had had initial conventional teaching. 
Computer-based learning projects in problem-solving 
were being prepared to make the student seek out and 
use information and ideas to solve particular problems. 

The resources of the computer would be used to help 
and at the same time monitor progress. If an area for 
experiment was chosen so that the elements and under- 
lying rules are known, e.g. electric circuitry, some 
types of chemical reactions or clinical diagnosis 
(assuming relations between diseases and symptoms can 
be stated in probabilistic terms), then the computer 
could teach adaptively. At Leeds there was now a frame- 
work for experiment, they knew what variables to work 
with and, hopefully, the instruction would be improved. 
Student performance data was on a scale which could be 
handled and which should give insight into the learning 
process. 

He finally expressed a hope that attention would be 
given to such small scale ‘depth’ projects for they may 
well provide a base for second generation CAI projects. 

Miss Ash thought that Mr. Hartley may be right in 
suggesting some limitations in setting up research projects 
but that his description of the teaching responsibility 
was a gross over-simplification. Teaching and learning 
involve far more than initial exposition, drill and 
practice and problem-solving. His approach would tend to 
place constraints on the educational processes and make 
them more rigid. She believed we should aim to extend 
the learner’s experiences, to broaden horizons rather 
than restrict them and that computers could help in this 
process. 

Mr. Broderick agreed with Miss Ash and suggested that 
one of the most rewarding applications of computers may 
be in the enrichment of the learning environment through 
simulation and game playing. For example, tremendous 
insight into chemical reaction could take place if a 
student were able to define two compounds and a set of 
physical conditions and observe the reactions taking 
place in slow motion on a graphic display. If we could add 
to this the facility for the student to manipulate the 
parameters of the system (the physical conditions) it 
could lead him to a better understanding of this aspect 
of chemistry. 

The Royal Liberty School had produced two gaming programs 
for experimental purposes, a business game and a town 
planning game. This work is preparatory to a deeper study 
of this aspect of CAI and whilst work is only at the 
early stages the teaching staff concerned were encouraged 
by the usefulness of the system and the students found it 
developed a better understanding of an otherwise 
‘abstract’ area of the curriculum by enabling them to 
learn heuristically. 

Dr. Adams pointed out that the machine was in fact well 
suited for teaching mechanical skills and that many of the 
problems facing society were at the level of people 
requiring such skills. These mechanical skills were the 
things that teachers claimed they could teach but the 
evidence was against it. 
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Professor Hansen gave three instances of fields in 
which the computer could play a major role. 

(i) Counselling work — he knew of cases where 
students preferred to use a computer because they 
could rely on obtaining accurate information. 

(ii) Computer Managed Instruction was becoming more 
prominent. It was certainly an area where 
information may be too much for teachers to 
handle. 

(iii) As a resource in complicated problem solving. 

Dr. Adams thought that the report of the N.C.E.T. study 
team had not taken enough notice of the design potential 
of the computer and instanced some work in lens systems 
that was being done in the Stater. 

Dr. Eraut brought the discussion back to the question 
of costs again and asked if the 6 hours of problem solving 
could justify the expense. 

Professor Hansen replied that the 6 hours problem 
solving could be taken separately and some students had 
done just this. Students who did the entire course did 
better. 

Dr. Rothkopf said that the costs of producing an 
hour’s instruction for Bell Telephone Company were about 
$10,000 and thought that Hansen’s figures should be 
examined very carefully. 

Professor Jones then introduced a short paper 
describing the system developed by him and J. M. Zelis 
at the University of Louvain and also used by le Corre 
in Paris. The system was part of a more general project 
intended to improve the study of mechanics in a General 
Physics course and altogether 80 students are involved. 
Each student works through a sequence of questions, 
answers and comments and these are linked together on a 
matrix so that they can be presented to the student in 
any order and in any way that is vanted. The answer the 
student gives leads to the next question or comment and 
the logic of the system ensures that no question which 
has received a correct response will be asked again. 

The work is done on the Bull-G.E C. time-sharing system 
and the following data is collected: 

For each student: order of questions 

number of questions 
time to respond to each question 
score for one sequence 
mean score for all sequences 
For each sequence: frequency for each question 
For all students: a mean value of the time for each 

question 

the time required for each sequence 
Time is also provided for each student to discuss his work 
with tutors. 

Miss Wallace presented a paper describing teaching 
undergraduate biology and medicine at the University of 
Glasgow. The most advanced work is a series of self- 
instructional units on endocrinology end renal disease 
involving the use of 35 mm. slides whii synchronised 



audio-tape. The student makes his response on a prepared 
response sheet. The team producing the work feel that 
although certain concepts can be taught by traditional 
programmed methods there is a strong case to be made for 
using a computer in the second stage of the process — 
transfer of knowledge to new situations. What is being 
proposed is a self-instructional system in medical 
education consisting of traditional lectures, tape/slide 
programmes (self-instructional), computer assisted 
instruction (self-instructional) and tutorials. 

The educational approach to the CAI presentation is 
algorithmic so that there will be several tracks through 
the work but there will also be certain ‘gates’ which 
must be passed. It is hoped to have a random access 
slide projector linked to the teletype which will supply 
the visual element. The slide projector will present 
summary captions from the tape/slide programmes and 
these captions will provide remedial loops. There are 
no plans at present for linking a random access tape- 
recorder to the system. The cost of producing the 
educational content of one hour of instruction will be 
close to £1000 — somewhat more than five times the cost 
of producing the first copy of tape/slide programmes. 

An investigation into the possibility of using the COTAN 
on-line desk system was being carried out and there would 
be co-operation with Culham laboratory in this field. 

Mr. Hodgson read a paper on the work of the Centre 
for Structural Communication. The researchers at the centre 
believed that computer-based learning systems suffer two 
major areas of neglect. These are: 

(a) Unrealistic estimates of the difficulty and 
magnitude of the task of computer compatible 
curriculum design and development — the ‘software’ 

gap- 

(b) A confusion in relating the sophistication of 
computer data processing to the actual level of 
educational significance which that data 
processing possesses — the ‘blinding by 
technology’. 

Work at the centre could make significant contributions 
to these areas and they felt that Structural Communication 
enabled the computer to be applied to areas of higher 
complexity in Bloom’s cognitive domain rather than to 
just knowledge, comprehension and application. The 
modular design of Structural Communication is easily 
implemented in CAI and CMI and several U.S. organisations 
(I.B.M., Westinghouse and the U.S. Navy) had used units. 

It had also been established that teachers and lecturers 
could be trailed to write study units in a time ratio of 
10 to 50 hours per student hour and that these units were 
meeting their objectives. Furthermore, the units already 
produced covered a number of academic fields (ranging 
from art appreciation to management and engineering) and 
for a variety of ages and abilities. 

Captain Huggett then described the system at H.M.S. 
Collingwood for training electrical technicians. The 
approach could be described as a systems approach with a 



O 

ERIC 







measurable input and output and the procedure adopted 
for evolving the work followed similar lines to those 
described in Hansen’s paper. There had been a considerable 
amount of work with programmed learning but tills was 
applied where it seemed appropriate rather than as a 
blanket device, i.e. they had a complete training package. 

Out of a twenty-five week course about half was programmed 



and there was considerable scope for revision as a new 
course started every two weeks. They were interested in 
using their present scheme as a testbed into which CAI 
could be fitted and in redeveloping their course so that 
the modules could be seen as a whole, i.e. they did not 
wish to go through the course, rewriting each module in 
turn but would like to develop the CAI aspect of all 
modules at the same time. 
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IN their recent report on computer based learning systems 
the NCET working group introduced the happy neologism 
“learning system analyst” to designate a person with an 
inter-disciplinary approach to the matters they discussed. 

I shall put on the hat of this new profession in tackling the 
more restricted topic of the present paper. 

From this point of view, it is evident that teaching is a 
form of control and that Teaching systems are built to 
control a learning process. This idea is developed in Section 1 . 
However, any controller is designed on the basis of a model 
for the controlled process. Hence, in Section 2, we take up 
several issues to do with learning and teaching models and 
relate these notions to the business of system design. 

To limit the scope of the discussion, it is assumed that the 
reader is familiar with the salient concepts of programmed 
instruction and systematic training which, in many respects, 
are precursors of the ideas involved in the (chiefly) computer 
assisted (CAI)* systems treated in the paper. It is also 
assumed that the reader is conversant with the main CAI 
display and response facilities (teletypewriters, CR tubes, 
light pens, function boards, computer controlled slide 
projectors) and the currently available response modes 
(multiple choice, constructed, word matching against 
synonyms, open ended etc.). Although programming 
languages and programme organization are vital constituents 
of CAI (and though many questions of this nature are 
relevant to the present argument) these matters are not 
considered since they lie in the province of other authors. 

Within this restricted field, the paper touches upon much 
of the research going on in Great Britain and the USA. No 
attempt has been made to review European and Russian 
developments, many of which are on a par with and some of 
which are in advance of the work that is mentioned. 

1. TEACHING AS THE CONTROL OF LEARNING 
1.1. Introduction 

A computer assisted teaching system (CAI system) is intended 
to control the learning process in an individual (the usual 
case), in a team or in a community. Proper as it is, this use of 
the word “control” has often led to crass misconceptions 
because “control” is rather narrowly interpreted. The 
required connotation is “guidance”. Control is not necessarily 
authoritarian. It may equally well be co-operative or catalytic. 
Moreover, tutorial control can be exercised even if the 
educational goal is underspecified (a point we take up in 1 .5 . 
For the moment consider the commonest case, where the 
educational goal is well defined). 

Now the symbolic instrument of control is a strategy; 
when it is lodged in a computing machine I shall call this a 
teaching strategy. But the influence of a teaching strategy is 
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always contingent upon its acceptance by the student and the 
actual comtrol of learning is frequently bilateral. Students 
learn continually, even in the absence of educators or their 
artifacts. In particular they direct their attention, choose 
goals to achieve (problems to solve) and partition them into 
subgoals (subproblems). This activity is governed by one or 
more of a class of learning strategies, certain qualities of 
which constitute the student’s learning set. Thus, any teaching 
strategy is in competition with learning strategies that already 
exist. It may be possible and expedient for the teaching 
system to suppress the student’s learning strategy and to 
introduce its own strategy as a surrogate. In other 
circumstances the teaching strategy is co-operative,* insofar 
as parts of it replace defective or missing segments of the 
student’s learning strategy. In other conditions again, the 
strategy followed by the coupled student-machine system is a ' 
compromise between the inbuilt teaching strategy and the 
strategy the student v/ould have adopted if left on his own. 

To illustrate these points, let us consider the sorts of control 
exercised in a number of teaching systems. 

1 .2 Representative Systems 

(1) Direct Individual Control. The computing machinery is 
equipped with a single teaching strategy. Excluding the trivial 
case, where the computer is used as a page turner** the 
execution of this strategy calls for more or less detailed 
performance information gleaned from the student to whom 
the machinery is coupled. Typical instances are CAI system" 
with history dependent branching programmes (decisions at a 
choice point may, perhaps, depend upon long sequences of 
past responses) and simple feedback controlled training 
systems such as Gaines’s, 1-2 Kelly’s, 3-4 Hudson’s, 5 Sime’s, 6 
or my own. 7-10b Here the level of task difficulty (for example, 



* It is worth making a distinction between the strategic co-operation, 
noted above, and the co-operation which goes on in any of the systems 
cited in 1.2.(1), (2) or (3). Even with a fixed strategy any of these 
systems perform an operation which we class as “increasing the task 
difficulty” or conversely “decreasing the task difficulty”. In reality, 
this is often a complicated operation based upon a model for the 
student’s learning process (as in Section 2 of the paper) and “decreasing 
task difficulty” is reducible to the canonical form “simplifying the 
problems posed by the task” or (equivalently) “partially solving these 
problems on the student’s behalf”. Such an operation is clearly 
co-operative though not in the strategic sense. Since all of the systems 
in 1.2. (1), (2) and (3) perform this operation all of them are partially 
co-operative systems. 

** Trivial only in the present context. Such arrangements may be 
quite useful but their value depends upon factors beyond the scope 
of this discussion; for example, the ease of recording responses, getting 
statistical data or updating administrative files. 
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the mean amplitude of a forcing function input in a tracking 
task, the pace cf operation in a perceptual motor skill, the 
withdrawal or delay of cueing information in an intellectual 
skill) is increased as the student’s proficiency increases and 
vice versa. The overall result is to balance the student at an 
operating point (in terms of task loading) that is 
predetermined to favour learning. 

(2) Adaptive individual control Performance information is 
used directly as in (1) but, in additio. , this information is 
evaluated and used to change one or more parameters of the 
teaching strategy. For example, the adaptive loop may change 
the operating point of (1) or it may change the weighting 
attached to several different variables which are adjusted to 
alter task difficulty or, as noted in (3), it may even change 
the form of the strategy. This category of systems is 
exemplified by Smallwood’s teaching programme and by 
more sophisticated training systems for perceptual motor and 
simple intellectual skills. (Gaines, Sime, and Pask. 10-12 ) 

(3) Conversational Systems. * The machine is provided with a 
class of strategies based upon a preliminary investigation of 
the learning strategies adopted by a population of students; 
this class is open ended; new strategies can be added as they 
are discovered. The machine is also furnished with 
information about the appropriateness of these strategies 
(for example, that Z 1 is effective if the student assembles the 
solutions to subproblems in the context of the problem as a 
whole, Z 2 is effective if the student solves problems like a 
puzzlist by stringing the subproblems’, methods into a more 
or less linear sequence). 

For his own part, the student also has a class of learning 
strategies from which he selects one at any moment. In a 
conversational system the student’s and the machine’s 
strategies are described and discussed (at a higher level of 
discourse than the problem and solution dialogue of straight- 
forward instruction) and certain propensities of the student 
are specifically tested (for example, his ability to see 
problems as a whole, his ability to adopt an algorithmic 
approach or, at a more pedestrian level, the interference 
characteristics of his intermediate memory). Often the test 
data evaluates properties of which the student is either 
unaware or imperfectly aware. Other things being equal, the 
machine allows the student to employ whatever strategy he 
has selected but (a) the student may have doubts about how 
to learn, in which case the machine makes a suggestion or 
(b) the chosen strategy may be quite inappropriate insofar as 
(in view of the measurements just made) the student would 
be unable to handle the task in the way he prefers. If so, the 
machine overrides the student, tells him why it is doing so 
and enforces a substitute strategy. Naturally, this process is 
repeated throughout the conduct of teaching so that, on 
average, a compromise is achieved between the strategies 
preferred by the student and thqse preferred by the machine. 
It is also possible to envisage the evolution of hybrid 
strategies. This comment delineates an important area for 
research. 

* Here, I am using the word “conversational” with its full logical 
meaning. Thus a quasi natural language computer terminal is not 
necessarily a vehicle for conversation just because it permits direct 
online communication with a human being. Conversation involves 
discourse at several levels; the higher levels accommodate statements 
that evaluate, arbitrate, criticise, command and select whatever is 
designated by the lower level discourse. 



Several systems have a genuinely conversational calibre. 
Some of Stolurow’s 13 systems do so (the conversation is 
phased into parts that determine the student’s characteristics, 
parts involved in strategy selection, and so on). Kopstein and 

Seidel, 14 " lsb at HumRRO are designing their system, IMPACT 
with me required properties (a pilot version has been put into 
operation); programmes like TASKTEACH 16 and PLATO 17 
in its enquiry mode have many conversational features; Uttal 
has devised a system of this type. Conversational systems 
have been used for skill instruction in my own laboratory 
(Pask, 18 Lewis and Pask 19 ) and we have completed a study of 
conversational interaction in simple problem solving which 
unequivocally demonstrate the efficiency of this mode of 
control. 

In one way the distinction between the adaptive system 
(Type 2) and the conversational system (Type 3) is fairly 
tenuous. If (as in Type 2) the parameters of a strategy are 
altered by the control loop, then a family of strategies is 
generated. Thus both Type 2 and Type 3 systems are based 
on a class of strategies. However, Type 2 strategic control 
depends only upon the machine’s interpretation of the 
student’s behaviour. In a Type 3 system the student’s 
interpretation of his own state is also taken into account (this 
is quite crucial) and the machine is required to “interpret the 
student’s interpretation” as well as his behaviour. Indeed, the 
cycle of student-machine “interpretations of interpretations” 
has no theoretical limit and the total system is, in principle, 
akin to a system of interpersonal interaction. (Bateson, 20 
Laing, 21 Brodey 22 ). 

(4) Game like systems. All of the systems so far considered 
are game like if they are properly instrumented; game like in 
the sense that the student plays with or participates in 
discourse with the machine. However, certain systems are 
game like in a different sense; the student is invited to 
participate in a game which (like a business or management 
game) simulates a situation he is required to learn about. Two 
categories of system are -worth distinguishing (a) The game is 
a fairly veridicial representation of reality. Though it often 
constitutes a useful training device this fact is incidental to its 
primary function. Some examples of the category are aircraft 
simulators, the Leviathan simulation of Sidney and Berenice 
Rome 22-24 at SDC (a very large, computer controlled, system) 
the medical diagnosis game 25 at Harvard and the SIMPOL 
system used, in my own laboratory, to simulate the 
managerial and resource allocation aspects of a police unit? 6-27 
(b) Games which are rigged up in order to teach someone (say 
an economics student) about the symbolic structure of his 
discipline (economics). Here, the paraphernalia of the game is 
specifically devised for instruction; economists are not 
generally required to play such games in real life (though, in 
the computer oriented environment of the future, they may 
be). Perhaps the best known system of this type is the 
Sumerian game 28 in which history students learn about the 
socio-economic structure of Mesopotamian civilization in 
3500 bc (there are several less colourful examples). 

(5) Group systems. The idea that students can be used to 
teach one another is by no means novel, and the concept has 
been refined in terms of group dynamics by Abercrombie 29 
and by Ackoff. 30 Broadly, the adaptive and self organizing 
capabilities of the students are used as part and parcel of the 
teaching system. In many ways it is very convenient to 
mechanize the interaction between the students by providing 



interfaces via which they communicate with one another or 
via which they gain access to data sources. In this context it 
becomes perfectly evident that the tutorial process must be 
catalyzed and guided (a) by assigning roles (such as A 
instructs B or C co-operates with D or E is the subject matter 
expert on X, F on Y); (b) by providing the channels of 
communication necessary to these roles; (c) by varying 
certain economic parameters of the system (in particular, the 
cost of communication and the gain obtainable as a result of 
successful performance). 

Quite a lot of small group psychology, from Bavelas 
onward, is relevant to this theme; for example, Lee Christie 31 
has for many years been alive to the potentialities of group 
learning systems (see Kelly for a useful review, slanted 
towards this point of view). More specifically, Osgood has 
used the PLATO system for group tuition; Glaser and Klauss 32 
have examined reinforcement schedules for teams and Lewis 
and I have carried out a series of experiments on machine 
controlled group learning. 33-35 

The last system will be briefly described, since it illustrates 
the group teaching paradigm in a simple but realistic form. 

We were teaching inductive inference. For this purpose we 
used an iterated form of Bruner, Goodenow and Austin’s 36 
conjunctive concept attainment task (a) the group had to 
tackle a sequence of over 100 “concepts”, i.e. any member 
had to handle a subsequence of evidence and to give a 
description, keyed into his console, of an unknown 
conceptual class; (b) the roles were (I) transmitter of 
information (concept exemplars), (II) receiver of the 
information (the man who sees the wood for the trees); 

(c) the communication channel conveying knowledge of 
(conceptual class) membership information was perturbed by 
varying amounts of “noise” (misinformation) so the students 
had to learn inference in noisy conditions; (d) the students sat 
at consoles in separate cubicles in which they received or 
transmitted concept exemplars and knowledge of membership 
information and via which they keyed their hypothesis and 
conclusions into external registers; (e) after each concept had 
been dealt with the controlling automaton (I) delivered 
knowledge of results, proficiency measures and updated a 
variable called “Bank Balance”, (II) allowed the students to 
express their preferences for different roles, (III) weighted 
these preferences as a function of the standing “Bank 
Balance” and allowed the receiving students to purchase 
co-operative communication links, (IV) assigned roles and 
sequenced the next set of moves, (V) set the level of 
misinformation so that a certain level of success could 
be reached. 

An initial study revealed that some groups learned better 
than others; indeed, a few failed to learn altogether. The 
successful groups could be characterized in several ways. For 
example, they engaged in an initial phase of objectively 
detectable co-operative activity; they traded off variety of 
behaviour (reduced by learning) for variety of communication 
and role structure; the entire group acted as a self-organizing 
system in the sense of von Foerster. 37 ' 39 Later in the series 
we introduced teaching strategies that acted upon the 
economic and role selection parameters of the system so that 
the conditions conducive to success obtained in any group, 
i.e. the controller was given a set of group teaching strategies 
and used these to catalyze favourable aspects of the 
developing group organization. The controlled groups were 
significantly more successful than the uncontrolled. Similar 



results were obtained in training a trajectory interception 
team but, although the latter work was done in the early 
1960s, the results are only available in report form. 40 

1 . 3 . Areas Omitted 

There are two major omissions from the list; both are 
important but neither is fully within the compass of this 
paper. (1) The computer is used as a tool (or laboratory) for 
teaching the student mathematics or programming. Notable 
instances are Feurtzig and Paperts LOGO 41 and James 
Thomas’s 42 work with TELCOMP (both addressed to school- 
children 10-12 years upwards). Here the computer only 
controls the student in the rather esoteric sense that any 
environment controls its inhabitants. Of course, the subject 
matter is irrelevant to the comment. The systems mentioned 
above are facilities (environments) that extend the student’s 
programming experience. By way of contrast, Seidel and 
Kopstein’s system at HumRRO is a computer controlled 
learning system that also (incidentally) teaches COBOL 
programming. (2) The computer is used to control 
community or school activities by differentially routing 
individuals through various educational tasks (for example, 
practical work, reading, programmed instruction). This is a 
promising field, so far chiefly repress.!; d by Flanagan’s 
PLAN project. 43 

1 . 4 . A Couple of Common Myths 

Even with these exclusions the systems listed in 1 .2 provide 
sufficient evidence to destroy the common myth that CAI 
is wedded in principle to the rather tedious paradigm of a 
question and answer routine. It is not. 

Before going on, I would like to discredit another myth 
(mentioned in 1 .1) which does a great deal to hamper 
development in CAI. The myth in question is that the 
educational goal of the control system must be fully and 
formally specified. Of course, the goal of the controller 
itself must be well defined: but that does not mean that we 
have to know the “right” answer to each question. There need 
not even be a “right” answer. It is still perfectly possible to 
design a heuristic device that encourages learning. 

The following hoary and whimsical example demonstrates 
the point. Suppose you make utterances Xi, x 2 . . . (all of which 
are members of a set X) to a friend and that the friend replies 
with utterances, gestures or grimaces , y 2 . . . (selected from 
a set Y). A very simple minded control scheme amounts to 
rewarding certain of the x, y relations with a nod of approval, 
i.e. you choose a mapping 0 q : X->Y and reward all y such 
that y = 0 Q (x). Here, the mapping 0 Q is the crux of a fully 
specified educational goal. Equally well, however., you might 
choose to reward any consistent relationship, i.e. your friend 
could, unknown to you at the outset, select any 0, not your 
0 o and you would reward him provided his usage is regular 
thus encouraging him to learn some relation. Of course, you, 
as the controller, have a well-defined goal (mathematically it 
is of the form: maximize— dH(X, Y)/dt where H(X, Y) is the 

contingent entropy of the selections from X and Y but you do 
not have a fully specified education goal as you did in the 
first variant of the experiments. 

The principle involved is neither restricted to trivial 
systems nor to simple minded reinforcement procedures. For 
example, it is easy to set up an adaptively controlled system 
in which the student is provided with more items to classify 



as he shows evidence of producing any self consistent and 
informative (efficient) scheme of classification (the student’s 
reaction is interesting in its own right; people who like to 
innovate find life in this libertarian teaching system 
exhilarating; those who do not, find it thoroughly 
disturbing). 44 There is no reason why the same ideas should 
not be applied to much more complex teaching situations 
and a moment’s contemplation indicates that similar notions 
are implicit in the operation of any open ended 
conversational system. 

1.5. The Evaluation of Control Methods 

The categories of control listed in 1 .2 have psychological 
cogency; they correspond to recognizable situations or 
relations between the student and the machinery. They are 
also quite realistic. Given a skill or a body of subject matter 
to be taught, and given a description of the student 
population it is possible to come up .with estimates of the cost 
of each sort of control in terms of hardware and software. 
Further, a good deal is known about the behaviour and 
relative efficiency of systems Type 1, 2, 3, and 5 (when a 
skill and a student are specified).* 

Evaluation is possible because the control methods are 
specified relative to models for learning; hence the methods 
may be evaluated in respect to real systems with which the 
model in question is identifiable to form a theory of learning. 
The nature of these theories is taken up in Section 2. But we 
comment, at this juncture, that these theories are 
educationally relevant insofar as they treat subject matter 
organizations and modes of cognition that are 
characteristically in the human domain and with which all 
teachers are familiar. Hence they are not amongst the simplest 
(Sorts of learning theory which appeal to experimental 
phychologists because of their elegance and structural 
parsimony; the simplest theories are often (and perhaps 
rightly) discounted on the grounds that they have little 
relevance to education. The present batch of theories are 
proof against this criticism (and so are the system evaluations 
derived from them). However, both the theories and the 
evaluations can be attacked from a different quarter. 

The fact is, the theories are most readily testable when the 
corresponding learning models are identified with laboratory 
situations. There are several reasons why this is so; the models 
themselves are workable with respect to small (though, as 
above, representative) chunks of mental activity. At an 
experimental level, the effect of individual differences 
(between subjects) becomes embarrassingly large if the 
experiment is unduly prolonged (by “pure” psychology 
standards the experiments are fairly lengthy; even so they do 
not often last more than a day or two). Finally, there is a 
purely practical reason why the control systems have been 
tried out in miniature. Most of the work in this field has been 
conducted in laboratories which (until recently at any rate)** 
have had special purpose on line control gear rather than on 
line equipment controlled by a general purpose computer 
(as a half-facetious, half-serious aside, workers having “big” 
CAI interfaces seem to have been preoccupied with getting 
them going rather than finding out what they do). Under 
these circumstances the cost of experimentation is appreciable 
and it is necessary to choose experimental situations that are 
as small as possible. 

Because of all this, it is perfectly possible to maintain that 



the theories and consequently the control methods attached 
to them have been tested in conditions that are educationally 
picayune. That is a caveat which must qualify all of the 
comments in the next few paragraphs and it may or may not 
be viewed as a damning one. My personal conviction 
(supported by the argument in Section 2) is that the 
educational relevance of the theory and the test situation are 
of primary importance and that the time scale of the 
experiment and the precise character of the problem solving 
activity do not matter too much. 

The following notes are grossly generalized summaries of 
the main findings. 

If " skill or body of knowledge is unstructured, i.e. if 
student are unable to say how they learn it after careful 
interrogation, apart from a statement to the effect that 
they engage in practice, then a fixed training routine (a 
feed forward procedure) appears to be as good as any other. 
Acquisition is just a matter of repetition. Type 5 control is 
useful in maintaining the student’s motivation and may be 
used to check his perseverance. 

Rather few skills are completely unstructured. Presumably, 
the skills (if any) acquired by simple conditioning are of this 
sort. In the later stages of learning (but not at the outset) 
many perceptual motor skills are unstructured. The same 
comment may apply to the later stages of learning 
computational skills (mental arithmetic) and possibly (though 
probably not) to the later stages of language acquisition. Rote 
learning, if it really took place, would be a matter of 
repetition but, unless precautions are taken to prevent him 
doing so, the student structures almost any list or catalogue, 
however meaningless it seems to be. 

In contrast a skill (body of knowledge) may be structured 
in the sense that (I) it can be represented as an hierarchy of 
subskills or (II) a goal-subgoal hierarchy or (III) an hierarchy 
of TOTE units or (IV) an hierarchy of concepts or (V) if 
learning can be conceived as the elimination of a finite 
number of Error Factors. If the skill is structured (and the 
student can be persuaded to see it in this way) it becomes 
possible to talk about the existence of learning strategies. 

Suppose that one such strategy has been selected on logical 
oi psychological grounds as a good strategy. Now, a Type 1 
Feedback control system designed relative to this good 
strategy is more effective than either (1) a fixed (feed- 
forward) routine based upon the same strategy or (2) Free 
learning. The advantage becomes marginal as (a) the 
acquisition of each component of the skill (body of 
knowledge) approximates “one shot” learning or (b) the set 
of possible strategies are uniformly “good”. 

Consider a family of learning strategies derived from a 
single good strategy and a Type 2 control system based upon 
this strategy family. In general, the Type 2 system is more 



* The evaluation of Type 4 control depends ceitically on the material 
Sometimes a game like mode of instruction is mandatory. When a game 
is optional there is evidence to suggest that Type 4 control is a relatively 
effective method of instruction. 

** The picture in this country is as follows: Kay and S.ime have 
recently set up a computerised installation at Sheffield. Annett and 
Duncan at Hull, have an interface linked to an Eliott 903 which is just 
becoming operational. Cook, Hartley and Sleeman have a system with 
multiple hardware (KDF9, Modula R..I.) at Leeds. Gaines still relief 
chiefly upon special purpose equipment. So do I, though we are 
considering the economy of using a general purpose machine. It should 
be emphasised that the experimental requirements are fairly quirkish. 

A simple On line terminal is virtually useless for this purpose. 



effective than the corresponding Type 1 system, if either 

(1) there is interference between the acquisition of one part 
of the skill (body of knowledge) and some other part of it or 

(2) the student needs to build up a learning set in order to 
secure competent performance (“interference” and “learning 
set” may be interpreted within any of the frameworks (I), 

(II), (III), (IV), (V)). 

Suppose there is a class of learning strategies which may 
be adopted by different students (or the same student on 
different occasions). It may not be possible to single out a 
strategy as “good”; for example, because some student uses 
each strategy effectively, i.e. we could only say “that strategy 
is good for a student with certain characteristics”. In this case 
a Type 3 conversational control system is more effective than 
either a Type 1 or a Type 2 system designed on the basis of 
an arbitrarily chosen good strategy and is very much more 
effective than free learning. The merit of Type 3 appears to 
be greatest when the influence of cognitive fixity (getting 
stuck with an inappropriate strategy or selection) is most 
obtrusive. As noted in 1 .2 Type 5 control may be used to 
achieve Type 3 control, either in the context of a group of 
otherwise independent individuals or in the context of a team. 

The majority of evaluations are expressed in terms of trials 
or time to reach a criterial performance with some check 
upon retention. A few are expressed in terms of uncertainty 
reduction. A more telling measure might be the time to reach 
a given level of understanding as indicated by the extent to 
which the knowledge in question is locked into a cognitive 
network. Tests for such a property have been devised but 
have not been extensively used. 



1.6. Control Methods as Educational Facilities or 
Educational Subroutines 

There is little difficulty in isolating skills such as list learning, 
relation learning, series completion or concept attainment for 
which one of the control methods listed in 1 .2 is better than 
the rest, and to determine the cost of instrumenting it. 

However, the majority of curricula— in mathematics, 
statistics, history, etc., call for the inculcation of several skills. 
This comment also applies outside the academic sphere; in the 
instruction of many complex tasks. Here, the picture is a 
good deal clearer since the subject matter is not so 
conventionally (perhaps arbitrarily) ordered. 

In COMCEN operator training, for example, it is possible 
to justify the use of all five control procedures (listed in 1 .2) 
at various stages in the process. 45 This is true, even if we 
confine our attention to the code teleprinting skill which is 
one ingredient of a COMCEN operator’s repertoire. 

At the outset, there is a strong element of problem solving. 
The student has to form concepts to do with different parts 
of the keyboard and the acquisition of one demonstrably 
interferes with tlhe acquisition of another. A similar situation 
occurs at a later stage, when the student is mastering the 
many to one relationships entailed by upper-case and lower- 
case operation. With the “difficulty” variables chosen as 
cueing and pacing, a Type 1 feedback control (Section 1 .2) is 
empirically superior to mere practice and, in certain 
circumstances (if the student’s behaviour goes outside a 
limited range), the Type 2 (adaptive) control is still better. 

In our own system, COMOPTS 1 , the demand for Type 1 and 
Type 2 control is met by providing an overall feedback 
control teaching system in which the student is normally 



situated and an adaptive system into which he is routed for 
remedial practice. 

Up to 30 hours of conventional training (about 10 hours 
in the mechanized training system) there are several possible 
learning strategies. Since this is so, a conversational system 
(Type 3) is required (if only because we are unable to 
determine an appropriate teaching strategy on behavioural 
evidence alone). It is provided in a partially mechanized form, 
by a “programme” or “flowchart” in which the student 
substitutes values derived from the feedback trainer after 
each exercise block. Further substitutions are made to 
indicate the student’s preferences and to point out aspects of 
the skill that are causing peculiar difficulty. Following 
through the substituted flowchart, the student is directed 
either to alter parameters of the feedback machine, to 
rehearse the skill on the adaptive machine or to engage in 
other activities (which will, in the final version of the system, 
involve working through segments of a programmed text). 

After 30 hours of conventional training, the task is 
substantially homogeneous (from the student’s point of view) 
and the acquisition of further competence depends chiefly 
upon practice. Here, an open loop training system would be 
as good as any other (given only a minimum monitoring of 
performance). The main problem is to keep the student 
motivated as he grinds away. For this purpose, COMOPTS 1 
will make use of control methods Type 4 (simulation) and 
Type 5 (gaming), i.e. students will be periodically presented 
with realistic communication situations and periodically 
required to play partly competitive and partly co-operative 
games with one another via the teleprinter interface 
(augmented by a board that delivers commands and 
synchronizing signals). The frequency and the nature of these 
interventions are automatically scheduled as a function of 
performance which is measured during the communication 
act. These measurements provide sufficient information for 
monitoring purposes. At the moment, the monitoring is 
manual but it could be automated without difficulty. In a 
programme such as TASKTEACH (which is capable of 
teaching quasi intellectual skills, typically fault detection and 
maintenance), an instructor sets up the list which gives the 
system a structure akin to COMOPTS; within this framework 
the student is free to choose his own fate (conversational 
Type 3 control) or to invoke more restrictive procedures 
(resembling Type 1 and Type 2 control). 

Similar comments apply to PLATO, Kopstein and Seidel’s 
IMPACT and other large CAI systems. However, since these 
are generally used to instruct academic skills (statistics, 
computer programming and the like) the simple pattern is 
obscured. Statistics, for example, already has a conventional 
order imposed upon its instruction so that the plan for 
teaching the subject matter as a whole contains phases in each 
of which the control methods of 1 .2 are called into operation, 
generally with different parameter settings to achieve specific 
subgoals. In the context of a large CAI system the control 
methods of 1 .2 thus have the status of tutorial subroutines. 

It should be emphasized that the distinction between a 
tutorial subroutine and a full CAI system is made on the 
grounds of organization rather than size. To see this, notice 
that the Stanford System (Suppes and his colleagues) consists 
in a series of isolated subroutines which are called into 
operation at the discretion of a teacher or supervisor. Thus 
the “drill and practice” mode, used for teaching children to 
carry out arithmetical operations, is a large but simply 
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structured feedback (Type 1) subroutine; the “tutorial” 
system employed for logic instruction (and other purposes) 
is a mixed conversational and adaptive subroutine (a hybrid 
of Type 2 and Type 3). The Stanford system has, 
nevertheless, massive provisions for recording and evaluating 
data and the system itself is one of the largest and most 
liberally interfaced* in existence. Of course, if the external 
supervisor uses the data to select the subroutines or to adjust 
their parameters then the system ( including the supervisor) is 
a full CAI system. Without the supervisor it is not. 

1.7. Evaluation of the System as a Whole 

It can be stated with some confidence that a full CAI system 
is no less effective than a system of programmed instruction 
or a system of classroom instruction. The more cautious 
commentators, seeking a general evaluation of the art, have 
gone no further than this and have still managed to justify 
CAI on economic grounds for higher education, special 
civilian and military training (where instructional costs are 
high in any case), and for many industrial and governmental 
purposes. When relatively inexpensive systems (such as 
Bitzer’s) come into operation the cost of CAI will fall from 
about $2.4 per student hour to about $0.35 per student hour 
due, in no small measure, to large-scale production techniques 
and hardware innovation; this change will place CAI within 
the economic compass of the school system. 

But all of these estimates, predicated on “CAI is no worse 
than . . .”, may be unduly pessimistic. In view of the 
information available about the basic control methods 
(educational subroutines), it is not unreasonable to suppose 
that CAI may be “a great deal better than . . .” other 
techniques and, if so, any cost benefit argument in its favour 
should be more readily acceptable. 

Unfortunately, the evaluation of a total CAI system is 
beset by a number of difficulties. Some of these have already 
been mentioned; for example, the problem of coping with 
individual differences. However, there are five obstacles 
which deserve special attention. 

(1) Providing it has been competently designed, any CAI 
system whatever is bound to give a saving in time and effort. 
For example, we can claim that COMOPTS will train 
teleprinter students (up to 3 or 4 week level) in half the time 
taken by a conventional training routine. But we have a 
relatively vague idea about how much of this effect is due to 
the tutorial action of the system. Some of the benefit stems 
from an incidental rationalization of improvident training 
techniques; in specifying any CAI system an efficient 
planning procedure is forced upon the designer. In particular, 
the CAI system is far more flexible and individualized and the 
greatest organizational advantage is gained when it is possible 
to get rid of the class attendance concept (so that the system 
can be used optimally with respect to the pupils). The Army 
Operational Research Group 46 in a much earlier study, 
predicted a similar enhancement in training' efficiency (for 
morse training) on these grounds alone; without reference 
to CAI as such. 

Needless to say, the same comments apply with even 
greater force to CAI systems used in the academic sphere. 

The enhancement due to organizing instructions and avoiding 
. time wastage is particularly great if, with Eide, 47 we count the 



* For example, the student receives spoken messages in addition to 
visually displayed material. 



students’ time as one of the most costly educational resources 
(there is a very strong case for doing so at the national level). 

(2) Any competent CAI system will work because of the 
Hawthorne effect. It is difficult to discount the influence of 
novelty in an experimental situation. Desirable though the 
improvement may be, it is not directly relevant to the tutorial 
influence of CAI. 

(3) Whereas it is easy enough to find standard conditions 
for controlling small-scale experiments (free learning or a 
fixed training routine) it is not so easy to select a control 
condition for a full CAI system (glibly CAI performance is 
compared with the performance of “conventional teaching”, 
but what is “conventional teaching”? It is fairly well defined 
for perceptual motor skills but not for academic skills). 

(4) Individual differences are not only large (a point made 
already in 1 .5) but specific to the motivational variables that 
appear in the fully-built CAI system. 

(5) The final and fundamental difficulty is that no one 
really knows what to measure (as evidenced by the fact that 
vast amounts of latency and response data have been collected 
on tapes, though very little of it has been used for evaluation). 
Just as the functional evaluation of an educational subroutine 
calls for a learning theory (which, amongst other things, tells 
the experimenter what to measure) so the evaluation of an 
entire CAI system calls for an educational theory. At the 
moment no adequate theory exists, though several relevant 
developments are noted in Section 2. 

Thus, at the moment, we are in the position of a programmer 
equipped with a collection of working and evaluated 
subroutines who is anxious to evaluate a programme (the full 
CAI system) constructed from these components. It would 
be naive to suppose that the evaluation of the whole (in the 
educational case) is the sum of the evaluations of the parts. 

In some instances, useful limits can be set up in terms of 
subroutine evaluation but a general appraisal of CAI seems to 
rest, as suggested above, upon an educational theory. We 
return to this point in Section 2.8. 

1.8. Critical Statement 

It is true that computers have often been used (under pressure 
to use the things somehow) in an unimaginative, narrow or 
downright stupid fashion. But this should not lead to a 
condemnation of the entire field: computer control of 
learning has great and ‘diverse potentialities, some of which 
have already been realized. 

The general development of computer control does, 
however, entail paying more attention to psychology and 
logic than it is currently fashionable to do. 

To take just one aspect* of the psychological point, most 
of the varieties of control listed in 1 .2 call for an adequate 
interface (which is certainly not a teletypewriter) and an 
adequate language. These questions are really in the province 
of other authors at this symposium; I only wish to make two 
points which illustrate a broad contention: (1) any 
conversational system must have an interface and a language 
which allows the student to state what he intends to do, not 
only as a plan for action but also as a plan for learning; (2) by 
the same token, the interface and language must allow the 
student to evaluate prescriptions and descriptions either to 
do with the task itself or with the process of learning it. One 
of the most useful evaluations is a subjective uncertainty 

* We return to it in the section on learning models. 
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measure, preferably obtained with a Shuford 48 type of 

scoring function as mechanized by Baker. 49 * 

These facilities are mandatory adjuncts to quite simple 
studies of teaching and it is disheartening to find elaborate 
computerized systems which lack them altogether. Needless 
to say, the systems cited in this paper, especially Seidel and 
Kopstein’s system, do have such facilities. So do many 
European and Russian systems which are excluded from the 
survey. 

On the issue of logic, there is an understandable tendency 
to see things through the eyes of computation technology. 
Logic is parochially (and naively) identified with 
propositional and first order predicate calculus. But, insofar 
as a logic of teaching is a logic of conversation, this is an 
insufficient framework. It is at least necessary to synthesize 
a scheme from the framework as it stands and (a) a logic of 
relations (the higher order predicates); (b) a logic of 
commands (Rescher so and von Wright sl ); (c) a logic of 
parallel (rather than sequential) organizations, for example, 
using Holt’s scheme ; s2 (d) a context logic (in the sense of 
Kotely S3 ); (e) a logic of distinctions and form (Spenser 
Brown S4 ) and (f) a logic of games and metagames in the sense 
of Howard. ss Some efforts have been made in this direction, 
notably by the social anthropologists who face much the 
same problem as “learning system analysts”. It is plain silly 
to pretend the problem does not exist, to give up in despair 
or to bludgeon away with techniques that serve well enough 
for describing mechanical computation (as carried out by 
present-day computers) but are unfitted to adumbrate the 
dynamics of mentation. 

2. MODELS FOR LEARNING AND TEACHING 

2.1. Overview 

Any controller is designed according to a model which can 
be interpreted and identified as a theory of the controlled 
process. In the case of teaching (the control of learning) this 
theory is a learning theory and it represents either the student 
alone or the student coupled to a specific teaching system. 

We emphasize that the underlying model is the designer’s 
model of the process; it is he, the designer, who interprets it 
as a theory. Furthermore, it is he who carries out experiments 
to verify the theory and it is he who, if necessary, alters the 
model to fit the facts. 

In contrast, models may also be used inside systems. Here 
the teaching device is provided with an internal model which 
it is built to interpret as a theory of the student with respect 
to the educational goal. It uses the model to determine its 
next move, it performs experiments to validate the model, 
and, if necessary, it modifies the model. In other words the 
system learns about the student and builds up an internal 
representation of the student state. The distinction between 
(designer’s) models of the system and (system) models in the 
system is non trivial when the system acts in this fashion. All 
conversational systems and most adaptive systems do so. 

(It is of interest to note that the system might start from 
scratch and build up a student model de novo. In practice this 
process would take far too long and even if it could be 
instrumented would clearly constitute an uneconomic design.) 



* Other techniques, such as a mechanised version of the Kelly grid, 
could be introduced with advantage. 



2.2. The Scope of the Models 

We have talked of learning theories and student models as 
though these things were neat packages. In fact, there are 
several separate representations which form part of any model 
and which are given varying amounts of emphasis. 

(1) A representation of the subject matter to be instructed. 

(2) A representation of the educational goal. In the simplest 
case this is a statement of the looked for terminal behaviour 
but frequently there are cognitive components in the 
specification. 

(3) A representation of the initial state of the student 
when he enters the teaching system i.e. of his current level of 
competence, his attitudes and possibly his personality. 

(4) A representation of the current state of the student 
(3) above providing a statement of the “initial conditions”) 
Generally, this part of the model is far more detailed and is 
mainly restricted to a representation of the student 
competence and his learning strategies (if attitudes, 
personality etc. appear in the model at all they are assumed 
to remain unchanged by learning). 

(5) A representation of the teaching system including its 
teaching strategies. 

All of these representations are present (perhaps in an 
embryonic form) within competent (designer’s) model of the 
student and his instruction. Models in the system are set 
within the framework afforded by (1), (2) and (3) but 
(insofar as the internal model is changed as a function of 
experience) the internal model itself is restricted to 
components (4) and (5). 



2.3. Some Types of Learning Theory 

Confining our attention to components (3) and (4) of 2.2, 
there are several types of learning model available to the 
designer of a CAI system. (Many of these are only 
interpretable on the “subroutine” scale considered in 
Section 1.2). 

In order to appreciate the state and potentialities of 
CAI it is necessary to review the major types of learning 
theory and to comment upon their ranges of application. 

As a prefatory remark the various theories are often 
complementary to one another; the differences are mainly 
philosophical. At a pragmatic level the recommended teaching 
strategies tend to be quite similar (as they should be! the 
theories are theories of the same system!). Nevertheless, one 
point of view may be very much more useful than the rest 
when it comes to modelling a particular situation. 

(1 ) Behavicuristic Models. The operant conditioning 
approach of Skinner was largely responsible for “linear” 
small step programming (a teaching paradigm still at the roots 
of many CAI systems). More liberal concepts of behaviour 
shaping have led to the “Mathetics” type of programme 516 
(which consists in large frames, eliciting many covert and 
overt reponses, each one resembling a well planned 
advertisement).* The S-R-operations (apart from basic 
conditioning and association) are chaining, discrimation and 
generalization and these operations are evoked in sequence in 



* In terms of this paper, both Skinnerian and “Mathetical” techniques 
are “feedforward” techniques. The student received a reinforcing 
feedback but feedback signals are not employed to modify the sequence 
or type of instruction. 
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order to produce (by hypothesis) such a juxtaposition of 
associative connections that the criterion behaviour can be 
achieved. 

The. main attraction of this theory is its simplicity, 
atomicity and quasi physical calibre. Its main disadvantages 
are (a) that the structuring of the task (representations (1), 

(2) and (3) of 2.2) is outside the model and is catered for 
informally (the matter is left to the designer’s discretion). 
(b)The representation of such entities as concepts, rules and 
plans ((4) and (5) of 2.2) is exceptionally cumbersome and 
may even be impossible. 

(2) Stochastic learning theories, especially the stimulus 
sampling theory of Estes and his colleagues. 57 " 58 Stochastic 
models are admirable predictors of the statistical properties 
of simple behaviour, for example, of response probabilities 
(strictly, these are properties of an ensemble of students but 
they may be interpreted as properties of processes which are 
ubiquitous and thus represented in any student). Some typical 
and interesting applications to teaching are Dear and 
Atkinsom; 59 model for concept learning and Matheson s* 50 use 
of a stimulus sampling model in combination with the 
technique of dynamic programming to prescribe optimal 
teaching strategies for certain (more or less repetitions) tasks. 
The control and evaluation procedures in Smallwood’s 61 CAI 
system are based, primarily, upon stochastic learning theory 
and this scheme, though not so highly developed as 
Matheson’st, is applicable to a much wider range of subject 
matter. Hence the stochastic models have much to 
recommend them. Their disadvantages are those voiced in 
connection with the behavioural models and, in addition, the 
limited tutorial relevance of the statistical properties they 
predict so well. 

(3) Error Factor Theory. Within this (more or less 
behaviouristic theory) learning to do something is regarded as 
the elimination of the influence of Error Factors which 
prevent it being done and which are responsible for classes of 
mistaken responses. Error Factor Theory 62 is particularly well 
suited to designing feedback and adaptive control systems for 
perceptual motor and problem solving skills. It has the great 
advantage of imaging the response process as potentially 
multidimensional and the vector of difficulty variables 
(Section 1.2) can be placed in direct correspondence with the 
set of Error Factors. The forms of learning predicted by Error 
Factor theory and simple reinforcement theory are somewhat 
different but the recommended training strategies are much 
the same (Error Factor theory yielding consonant but more 
detailed recommendations). 

(4) Functional Theories. Enough is known about certain of 
the subsystems involved in learning to model them 
meaningfully as mechanisms (rather than “Black Boxes”).* 
This is especially true of the functionally demarcated 
subsystems “Immediate” (short term) “Working” 
(Intermediate) and Long Term memory, of which the first 
two are fairly well investigated. 



* In Ashby’s sense; modelling the input output characteristics or 
transfer function of a system. 



The characteristics and limitations of immediate memory 
are familiar from work in pure experimental psychology; 
clearly these limitations should be respected in the design of 
any CAI system (as in the design of any ether man machine 
interface). The working memory system is less well 
documented but is especially germane to teaching since it is 
in this system that skills and bodies of knowledge are 
integrated into structures and it is here that interference and 
positive transfer of training take place. 

There are various models for the system, but Atkinson 
and Shiffrin’s 64 is well validated and is stated in a form 
applicable to CAI. The working memory is akin to a 
programme embodied in a working storage which serves as 
a general purpose computer that may be programmed in 
several ways. Each way is an information control procedure; 
for example, one procedure is a rehearsal buffer, another is 
a sort of push down list processor, another is a procedure 
for transfer to long term memory. Fiegenbaum’s EPAM 
programme, 65-66 a computer simulation of working memory 
viewed as a discrimination and association network, is a 
further model of this type and is also well validated. Given a 
theory based on one of these models, it is possible to 
recommend very detailed teaching strategies. Several of these 
have been tested as part and parcel of the experimental effort 
connected with the modelling. It should be noted that the 
teaching strategies derived from functional theories are 
generally compatible with those derived from behavioural 
theories. But the functionally derived strategies are far more 
sophisticated. 

(5) Objective Informational Theories (Wattanabe, 67 
von Foerster, 73 Pask 68 " 72 ). The word “information” 

is used in its technical sense (selective information, measures 
of information) and learning is viewed as a reduction in 
relevant uncertainty which is influx ced by such variables as 
the student’s information loading and the informational 
redundancy of the materials presented to him. “Objective” 
implies that the informations, uncertainties, etc., in question 
are those of an outside observer, not of the student. On the 
one hand such objectively measured uncertainties are clearly 
related to statistical indices such as response probabilities 
(regard stimuli as signs selected from an input alphabet and 
responses as output selections). Thus the informational 
theories of learning are. related to the behavioural theories. 

On the other hand, the information measures, though obtained 
over sets of behavioural alternatives, may be held to estimate 
the student’s uncertainty. Insofar as they do so a bridge is 
established between behavioural theories and subjective 
information theories and via these between behavioural 
theories and cognitive theories of learning. 

(6) Subjective Information Theories. The information 
measures are computed from subjective estimates of degree 
of belief (or doubt) obtained by the methods of Shuford, 

Baker and others.* These workers have developed a learning 



* Essentially in a game like situation where the student is assigned a 
score dependent upon, his numerically asserted degree of belief in 
several alternatives. His assertion is constrained to secure “probability” 
numbers which sum to one over all of the alternatives. The scoring 
function employed is such that, over a sequence of trials, the student’s 
mathematical expectation of score is maximised if, and only if, his 
asserted degree of belief equals his real degree of belief. 



theory which is comparable with an objective theory and have 
used it to prescribe various teaching strategies. 

(7) Structural Theories. The developmental psychologies of 
Piaget and Luria 74 are structural theories insofar as they 
postulate certain mental structures which dominate the psyche 
at different ages. At the level of individual learning (rather 
than development) there are several theories of a like kind; 
Gagne, 75 ' 76 for example sees the acquisition of a skill as the 
assembly of an hierarchical structure of capabilities. Learning 
occurs when a pair or more of subordinate capabilities are 
wedded together by an instruct! n that combines them in 
some way; but the instruction will only be effective if the 
requisite subordinate capabilities are already established. For 
dealing with a body of knowledge (rather than a skill) the 
“capabilities” are replaced by “concepts” (given a behavioural 
emphasis a concept is a capability to classify and respond). 

(8) Organizational and Cybernetic Theories. These are 
structural theories in which the basic units built into an 
hierarchical structure are goal directed, problem solving or 
control systems (either contained within the organisation or 
acting partly through the environment. The theory has a 
cognitive aspect; since “aiming for a goal” or “solving a 
problem” explicitly involves a state of knowing. Miller, 
Gallanter and Pribram 77 proposed a TOTE (test operate test 
exit) unit as the goal directed building block; essentially this 
is a piece of programme which corresponds to a contingent of 
IF, THEN, ELSE, statement. A plan or a concept is conceived 
as a nested structure of TOTE units. 

I have pointed out that in order to build up or modify 
such an hierarchy (i.e. in order to talk about learning) it is 
necessary to invoke an independent hierarchy of control (or 
problem solving of problem solving). Because of this any 
Cybernetic learning model is heterarchical 78-79 c 



(9) Programmatic Cognitive Theories. In general these turn 
out to be Cybernetic theories although the basic units are 
specified simply as information processing programmes, the 
execution of which may be expected to produce a state of 
knowing and the consciousness of a plan, hypothesis, 
concept or strategy.* All artificial intelligence models which 
purport to simulate human learning belong to this class. Since 
the models are very complex and varied, it is impossible to 
convey the gist of them in a few words but the subject has 
been reviewed in another paper. 80 Bennett and Hodges 
“Structural Communication” or “Systematics” type of 
programming is based on a cognitive modelf 1-82 informally 
so at the moment, but a formal treatment is clearly possible. 

It is also quite clear that although systematics programmes 
can be administered by textual methods, they are really fitted 
to CAI administration and have the status of CAI systems. 

(10) Other Cognitive Theories. Several perfectly respectable 
cognitive theories do not fit easily into the currently popular 
programmatic framework. Two good examples are Festinger’s 83 
cognitive dissonance theory and Kelly’s personal construct 
theory. 84 Both of them have potential value in connection 



* Unless the entity is specifically tagged as a reflexive or autonomous 
unit. 



with certain aspects of learning and teaching (notice for 
instance the close relation' between the “cognitive fixity” of 
1.2 and “cognitive dissonance”). 

(11) Statistical Cognitive Theories. Guildford 8s and 
Bunderson’s 86 factor analytic studies of learning showed that 
different mental processes (aptitudes) are important at . 
different stages in learning the same task. Clearly, these 
“processes” have a statistical calibre akin to the response 
“processes” in a stochastic model apart from the fact that 
they are cognitive rather than behavioural entities. A body of 
hypotheses frames in terms of there variables can be usefully 
interpreted as a theory (such a construct is used in the 
IMPACT model). 

(12) Theories of representation and learning type, notably 
Bruner’s 87 theory of ikonic, mnemonic and automatic 
representation and the linguistic theories of Vygotsky, 88 
Luria and their pupils. 

2 . 4 . Models and Theories of Learning 
The learning theories of 2.3 are concerned with the 
components or representations (2) and (4) of Section 2.2 
(namely of the student’s initial state and his current state). 
Given a specific task or body of knowledge it is fairly easy to 
adjoin a representation of the subject matter (component (1) 
of 2.2) and of the educational goal (component (2) of Section 
2.2) to yield a descriptive and predictive (or explanatory) 
theory of learning the task in question. Clearly, the 
representations employed for this purpose must be compatible 
with the type of theory chosen and it is almost platitudinous 
to remark that the choice of a learning theory will depend 
upon the subject matter and the educational goal that an 
investigator has in mind. For example, the behavioural 
theories are simple and thus (in one sense) preferable. But it 
is difficult if not impossible to set up a subject matter 
representation which is compatible with such a theory if the 
subject matter entails plans, concepts and the like, in anything 
more than a naive fashion.* In general, it is necessary to 
reach a compromise between theoretical simplicity and 
realism of subject matter representation and the structural 
and Cybernetic theories are peculiarly useful in this respect. 

Although the models of Section 2.3 are chiefly descriptive 
(and predictive) models, they can all be used in a prescriptive 
fashion** to yield recommendations for teaching; either a set 
of teaching strategies or a teaching system. Thus, we enquire 
what operations would be necessary to achieve the educational 
goal (with respect to a given body of subject matter and a 
given student model) and specify a teaching model capable 
of performing these operations. The requirement may, of 
course, be more or less refined; for example, we might 
demand a set of operations to achieve the goal as fast as 
possible or set of operations that lead to a generalized rather 
than a specialized competence. But, in any case, the teaching 
model constitutes component (5) of Section 2.2 and, when 

* This is a personal bias. For instance, I do believe it is naive to 
regard a concept as no more than a class of stimuli that evoke some 
common response. 

**The distinction between the descriptive and prescriptive use of 
models was originally drawn by Kopstein. It has been employed 
extensively by Stolurow and myself. Stolurow also consider models 
that have little predictive power. Thus, my usage is somewhat different. 
All of the models cited in this paper, for example', have predictive or 
explanatory power. 



r Hansen gave three instances of fields in 
le computer could play a major role, 
tunselling work — he knew of cases where 
idents preferred to use a computer because they 
uld rely on obtaining accurate information, 
unputer Managed Instruction was becoming more 
ominent. It was certainly an area where 
formation may be too much for teachers to 
ndle. 

; a resource in complicated problem solving. 

ms thought that the report of the N.C.E.T. study 
d not taken enough notice of the design potential 
Dmputer and instanced some work in lens systems 
being done in tne States. 

it brought the discussion back to the question 
again and asked if the 6 hours of problem solving 
stify the expense. 

r Hansen replied that the 6 hours problem 
:ould be taken separately and some students had 
t this. Students who did the entire course did 



ikopf said that the costs of producing an 
istruction for Bell Telephone Company were about 
and thought that Hansen’s figures should be 
d very carefully. 

r Jones then introduced a short paper 
lg the system developed by him and J. M. Zelis 
niversity of Louv'm and also used by le Corre 
The system was part of a more general project 
l to improve the study of mechanics in a General 
'ourse and altogether 80 students are involved, 
dent works thiough a sequence of questions, 
and comments and these are linked together on a 

0 that they can be presented to the student in 

;r and in any way that is wanted. The answer the 
gives leads to the next question or comment and 
: of the system ensures that no qut .Uo»» which 
ived a correct response will be ask again. 

k is done on the Bl'i’. G.E.C. time-sharing system 
following data is collected: 

1 student: order of questions 

number of questions 
time to respond to each question 
score for one sequence 
mean score for all sequences 
l sequence: frequency for each question 
tudents: a mean value of the time for each 

question 

the time required for each sequence 
also provided for each student to discuss his work 
ors, 

llace presented a paper describing teaching 
iduate biology and medicine at the University of 
r . The most advance, 5 work is a series of self- 
icnal units on endocrinology and renal disease 
g th~ oi 35 mm. slides with synchronised 



audio-tape. The student makes his response on a prepared 
response sheet. The team producing the work feel that 
although certain concepts can be taught by traditional 
programmed methods there is a strong case to be made for 
using a computer in the second stage of the process — 
t) t sfer of knowledge to new situations. What is being 
proposed is a self-instructional system in medical 
education consisting of traditional lectures, tape/slide 
programmes (self-instructional), computer assisted 
instruction (self-instructional) and tutorials. 

The educational approach to the CAI presentation is 
algorithmic so that there wiii be several tracks through 
the work b t there will also be certain ‘gates’ which 
must be passed. It is hoped to have a random access 
slide projector linked to the teletype which will supply 
the visual element. The slide projector will present 
summary captions from the tape/slide programmes and 
these captions will provide remedial loops. There are 
no plans at present for linking a random access tape- 
recorder to the system. The cost of producing the 
educational content of one hour of instruction will be 
close to £1000 - somewhat more than five times the cost 
of producing the first copy of tape/slide programmes. 

An investigation into the possibility of using the COTAN 
on-iine desk system was being carried out and there would 
be co-operation with Culham laboratory in this field. 

Mr. Hodgson read a paper on the work of the Centre 
for Structural Communication. The researchers at the centre 
believed that cohiputer-based learning systems suffer two 
major areas of neglect. These are: 

(a) Unrealistic estimates of the difficulty and 
magnitude of the task of computer compatible 
curriculum design and development - the ‘software’ 
gap. 

(b) A confusion in relating the sophistication of 
computer data processing tc the actual level of 
educational significance which that data 
processing possesses — the ‘blinding by 
technology’. 

Work at the centre could make significant contributions 
to these areas and they felt that Structural Communication 
enabled the computer to be applied to areas of higher 
complexity in Bloom’s cognitive domain rather than to 
just knowledge, comprehension and application. The 
modular design of Structural Communication is easily 
implemented in CAI and CMI and several U.S. organisations 
(I.B.M., Westinghouse and the U.S. Navy) had used units. 

It had also been established that teachers and lecturers 
could be trailed to write study units in a time ratio of 
10 to 50 hours per student hour and that these units were 
meeting their objectives. Furthermore, the units already 
produced covered a number of academic fields (ranging 
from art appreciation to management and engineering) and 
for variety of ages and abilities. 

Captain Huggett then described the system at H.M.S. 
Collingwood for training electrical technicians. The 
approach could be described as a systems approach with a 



measurable input and output and the procedure adopted 
for evolving the work followed similar lines to those 
: described in Hansen’s paper. There had been a considerable 

amount of work with programmed learning but this was 
k applied where it seemed appropriate rather than as a 
blanket device, i.e . they had a complete training package. 

Out of a twenty-five week course about half was programmed 



and there was considerable scope for revision as a new 
course started every two weeks. They were interested in 
using the a present scheme as a testbed into which CAI 
could be fitted and in redeveloping their course so that 
the modules could be seen as a whole, i.e. they did not 
wish to go through the course, rewriting each module in 
turn but would like to develop the CAI aspect of all 
modules at the same time. 
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IN their recent report on computer based learning systems 
the NCET working group introduced the happy neologism 
“learning system analyst” to designate a person with an 
inter-disciplinary approach to the matters they discussed. 

I shall put on the hat of this new profession in tackling the 
more restricted topic of the present paper. 

From this point of view, it is evident that teaching is a 
form of control and that Teaching systems are built to 
control a learning process. This idea is developed in Section 1. 
However, any controller is designed on the basis of a model 
for the controlled process. Hence, in Section 2, we take up 
several issues to do with learning and teaching models and 
relate these notions to the business of system design. 

To limit the scope of the discussion, it is assumed that the 
reader is familiar with the salient concepts of programmed 
instruction and systematic training which, in many respects, 
are precursors of the ideas involved in the (chiefly) computer 
assisted (CAI)* systems treated in the paper. It is also 
assumed that the reader is conversant with the main CAI 
display and response facilities (teletypewriters, CR tubes, 
light pens, function boards, computer controlled slide 
projectors) and the currently available response modes 
(multiple choice, constructed, word matching against 
synonyms, open ended etc.). Although programming 
languages and programme organization are vital constituents 
of CAI (and though many questions of this nature are 
relevant to the present argument) these matters are not 
considered since they lie in the province of other authors. 

Within this restricted field, the paper touches upon much 
of the research going on in Great Britain and the USA. No 
attempt has been made to review European and Russian 
developments, many of which are on a par with and some of 
which are in advance of the work that is mentioned. 



always contingent upon its acceptance by the student and the 
actual control of learning is frequently bilateral. Students 
learn continually, even in the absence of educators or their 
artifacts. In particular they direct their attention, choose 
goals to achieve (problems to solve) and partition them into 
subgoals (subproblems). This activity is governed by one or 
more of a class of learning strategies, certain qualities of 
which constitute the student’s learning set. Thus, any teaching 
strategy is in competition with learning strategies that already 
exist. It may be possible and expedient for the teaching 
system to suppress the student’s learning strategy and to 
introduce its own strategy as a surrogate. In other 
circumstances the teaching strategy is co-operative* insofar 
as parts of it replace defective or missing segments of the 
student’s learning strategy. In other conditions again, the 
strategy followed by the coupled student-machine system is a-- 
compromise between the inbuilt teaching strategy and the 
strategy the student would have adopted if left on his own. 

To illustrate these points, let us consider the sorts of control 
exercised in a number of teaching systems. 

1 .2 Representative Systems 

(1 ) Direct Individual Control The computing machinery is 
equipped with a single teaching strategy. Excluding the trivial 
case, where the computer is used as a page turner** the 
execution of this strategy calls for more or less detailed 
performance information gleaned from the student to whom 
the machinery is coupled. Typical instances are CAI systems 
with history dependent branching programmes (decisions at a 
choice point may, perhaps, depend upon long sequences of 
past responses) and simple feedback controlled training 
systems such as Gaines’s, 1-2 Kelly’s, 3-4 Hudson’s, 5 Sime’s, 6 
or my own. 7 “ 10b Here the level of task difficulty (for example, 



1. TEACHING AS THE CONTROL OF LEARNING 
1.1. Introduction 



A computer assisted teaching system (CAI system) is intended 
{ to control the learning process in an individual (the usual 

' case), in a team or in a community. Proper as it is, this use of 

the word “control” has often led to crass misconceptions 
because “control” is rather narrowly interpreted. The 
required connotation is “guidance”. Control is not necessarily 
authoritarian. It may equally well be co-operative or catalytic. 
Moreover, tutorial control can be exercised even if the 
educational goal is underspecified (a point we take up in 1.5. 
For the moment consider the commonest case, where the 
educational goal is well defined). 

Now the symbolic instrument of control is a strategy; 
when it is lodged in a computing machine I shall call this a 
teaching strategy. But the influence of a teaching strategy is 
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Computer As c :»ted Instruction. 



* It is worth making a distinction between the strategic co-operation, 
noted above, and the co-operation which goes on in any of the systems 
cited in 1.2.(1), (2) or (3). Even with a fixed strategy any of these 
systems perform an operation which we class as “increasing the task 
difficulty” or conversely “decreasing the task difficulty”. In reality, 
this is often a complicated operation based upon a model for the 
student’s learning process (as in Section 2 of the paper) and “decreasing 
task difficulty” is reducible to the canonical form “simplifying the 
problems posed by the task” or (equivalently) “partially solving these 
problems on the student’s behalf”. Such an operation is clearly 
co-operative though not in the strategic sense. Since all of the systems 
in 1.2. (1), (2) and (3) perform this operation all of them are partially 
co-operative systems. 

** Trivial only in the present context. Such arrangements may be 
quite useful but their value depends upon factors beyond the scope 
of this discussion; for example, the eas« of recording responses, getting 
statistical data or updating administrative files. 
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the mean amplitude of a forcing function input in a tracking 
task, the pace of operation in a perceptual motor skill, the 
withdrawal or delay of cueing information in an intellectual 
skill) is increased as the student’s proficiency increases and 
vice versa. The overall result is to balance the student at an 
operating point (in terms of task loading) that is 
predetermined to favour learning. 

(2) Adaptive individual control Performance information is 
used directly as in (1) but, in addition, this information is 
evaluated and used to change one or more parameters of the 
teaching strategy. For example, the adaptive loop may change 
the operating point of (1) or it may change the weighting 
attached to several different variables which are adjusted to 
alter task difficulty or, as noted in (3), it may even change 
the form of the strategy. This category of systems is 
exemplified by Smallwood’s teaching programme and by 
more sophisticated training systems for perceptual motor and 
simple intellectual skills. (Gaines, Sime, and Pask. 10 " 12 ) 

(3) Conversational Systems. * The machine is provided with a 
class of strategies based upon a preliminary investigation of 
the learning strategies adopted by a population of students; 
this class is open ended; new strategies can be added as they 
are discovered. The machine is also furnished with 
information about the appropriateness of these strategies 
(for example, that Zj is effective if the student assembles the 
solutions to subproblems in the context of the problem as a 
whole, Z 2 is effective if the student solves problems like a 
puzzlist by stringing the subproblems’, methods into a more 
or less linear sequence). 

For his own part, the student also has a class of learning 
strategies from which he selects one at any moment. In a 
conversational system the student’s and the machine’s 
strategies are described and discussed (at a higher level of 
discourse than the problem and solution dialogue of straight- 
forward instruction) and certain propensities of the student 
are specifically tested (for example, his ability to see 
problems as a whole, his ability to adopt an algorithmic 
approach or, at a more pedestrian level, the interference 
characteristics of his intermediate memory). Often the test 
data evaluates properties of which the student is either 
unaware or imperfectly aware. Other things being equal, the 
machine allows the student to employ whatever strategy he 
has selected but (a) the student may have doubts about how 
to learn, in which case the machine makes a suggestion or 
(b) the chosen strategy may be quite inappropriate insofar as 
(in view of the measurements just made) the student would 
be unable to handle the task in the way he prefers. If so, the 
machine overrides the student, tells him why it is doing so 
and enforces a substitute strategy. Naturally, this process is 
repeated throughout the conduct of teaching so that, on 
average, a compromise is achieved between the strategies 
preferred by the student and thqse preferred by the machine. 
It is also possible to envisage the evolution of hybrid 
strategies. This comment delineates an important area for 
research. 



* Here, I am using the word “conversational” with its full logical 
meaning. Thus a quasi natural language computer terminal is not 
necessarily a vehicle for conversation just because it permits direct 
online communication with a human being. Conversation involves 
discourse at several levels; the higher levels accommodate statements 
that evaluate, arbitrate, criticise, command and select whatever is 
designated by the lower level discourse. 



Several systems have a genuinely conversational calibre. 
Some of Stolurow’s 13 systems do so (the conversation is 
phased into parts that determine the student’s characteristics, 
parts involved in strategy selection, and so on). Kopstein and 

Seidel, 14 ' 1 5b at HumRRO are designing their system, IMPACT 
with tne required propemes (a pilot version has been put into 
operation); programmes like TASKTEACH 16 and PLATO 17 
in its enquiry mode have many conversational features; Uttal 
has devised a system of this type. Conversational systems 
have been used for skill instruction in my own laboratory 
(Pask, 18 Lewis and Pask 1 *) and we have completed a study of 
conversational interaction in simple problem solving which 
unequivocally demonstrate the efficiency of this mode of 
control. 

In one way the distinction between the adaptive system 
(Type 2) and the conversational system (Type 3) is fairly 
tenuous. If (as in Type 2) the parameters of a strategy are 
altered by the control loop, then a family of strategies is 
generated. Thus both Type 2 and Type 3 systems are based 
on a class of strategies. However, Type 2 strategic control 
depends only upon the machine’s interpretation of the 
student’s behaviour. In a Type 3 system the Student’s 
interpretation of his own state is also taken into account (this 
is quite crucial) and the machine is required to “interpret the 
student’s interpretation” as well as his behaviour. Indeed, the 
cycle of student-machine “interpretations of interpretations” 
has no theoretical limit and the total system is, in principle, 
akin to a system of interpersonal interaction. (Bateson, 20 
Laing, 21 Brodey 22 ). 

(4) Game like systems. All of the systems so far considered 
are game like if they are properly instrumented; game like in 
the sense that the student plays with or participates in 
discourse with the machine. However, certain systems are 
game like in a different sense; the student is invited to 
participate in a game which (like a business or management 
game) simulates a situation he is required to learn about. Two 
categories of system are-worth distinguishing (a) The game is 
a fairly veridicial representation of reality. Though it often 
constitutes a useful training device this fact is incidental to its 
primary function. Some examples of the category are aircraft 
simulators, the Leviathan simulation of Sidney and Berenice 
Rome 22 ' 24 at SDC (a very large, computer controlled, system) 
the medical diagnosis game 25 at Harvard and the SIMPOL 
system used, in my own laboratory, to simulate the 
managerial and resource allocation aspects of a police unit? 6-27 
(b) Games which are rigged up in order to teach someone (say 
an economics student) about the symbolic structure of his 
discipline (economics). Here, the paraphernalia of the game is 
specifically devised for instruction; economists are not 
generally required to play such games in real life (though, in 
the computer oriented environment of the future, they may 
be). Perhaps the best known system of this type is the 
Sumerian game 28 in which history students learn about the 
socio-economic structure of Mesopotamian civilization in 
3500 bc (there are several less colourful examples). 

(5) Group systems. The idea that students can be used to 
teach one another is by no means novel, and the concept has 
been refined in terms of group dynamics by Abercrombie 29 
and by Ackoff. 30 Broadly, the adaptive and self organizing 
capabilities of the students are used as part and parcel of the 
teaching system. In many ways it is very convenient to 
mechanize the interaction between the students by providing 
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interfaces via which they communicate with one another or 
via which they gain access to data sources. In this context it 
becomes perfectly evident that the tutorial process must be 
catalyzed and guided (a) by assigning roles (such as A 
instructs B or C co-operates with D or E is the subject matter 
expert on X, F on Y); (b) by providing the channels of 
communication necessary to these roles; (c) by varying 
certain economic parameters of the system (in particular, the 
cost of communication and the gain obtainable as a result of 
successful performance). 
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Quite a lot of small group psychology, from Bavelas 
onward, is relevant to this theme; for example, Lee Christie 31 
has for many years been alive to the potentialities of group 
learning systems (see Kelly for a useful review, slanted 
towards this point of view). More specifically, Osgood has 
used the PLATO system for group tuition; Glaser and Klauss 32 
have examined reinforcement schedules for teams and Lewis 
and I have carried out a series of experiments on machine 
controlled group learning. 33-35 

The last system will be briefly described, since it illustrates 
the group teaching paradigm in a simple but realistic form. 

We were teaching inductive inference. For this purpose we 
used an iterated form of Bruner, Goodenow and Austin’s 36 
conjunctive concept attainment task (a) the group had to 
tackle a sequence of over 100 “concepts”, i.e. any member 
had to handle a subsequence of evidence and to give a 
description, keyed into his console, of an unknown 
conceptual class; (b) the roles were (I) transmitter of 
information (concept exemplars), (II) receiver of the 
information (the man who sees the wood for the trees); 

(c) the communication channel conveying knowledge of 
(conceptual class) membership information was perturbed by 
varying amounts of “noise” (misinformation) so the students 
had to learn inference in noisy conditions; (d) the students sat 
at consoles in separate cubicles in which they received or 
transmitted concept exemplars and knowledge of membership 
information and via which they keyed their hypothesis and 
conclusions into external registers; (e) after each concept had 
be» n dealt with the controlling automaton (I) delivered 
knowledge of results, proficiency measures and updated a 
variable called “Bank Balance”, (II) allowed the students to 
express their preferences for different roles, (III) weighted 
these preferences as a function of the standing “Bank 
Balance” ai o allowed the receiving students to purchase 
co-operative communication links, (IV) assigned roles and 
sequenced the next set or moves, (V) set the level of 
misinformation so that a certain level of success could 
be reached. 

An initial study revealed that some groups learned better 
than others; indeed, a few failed to learn altogether. The 
successful groups could be characterized in several ways. For 
example, they engaged in an initial phase of objectively 
detectable co-operative activity; they traded off variety of 
behaviour (reduced by learning) for variety of communication 
and role structure; the entire group acted as a self-organizing 
system in the sense of von Foerster. 37 * 39 Later in the series 
we introduced teaching strategies that acted upon the 
economic and role selection parameters of the system so that 
the conditions conducive to success obtained in any group, 
i.e. the controller was given a set of group teaching strategies 
and used these to catalyze favourable aspects of the 
O developing group organization. The controlled groups were 
! KL ( significantly more successful than the uncontrolled. Similar 
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results were obtained in training a trajectory interception 
team but, although the latter work was done in the early 
1960s, the results are only available in report form. 40 

1.3. Areas Omitted 

There are two major omissions from the list; both are 
important but neither is fully within the compass of this 
paper. (1) The computer is used as a tool (or laboratory) for 
teaching the student mathematics or programming. Notable 
instances are Feurtzig and Paperts LOGO 41 and James 
Thomas’s 42 work with TELCOMP (both addressed to school- 
children 10-12 years upwards). Here the computer only 
controls the student in the rather esoteric sense that any 
environment controls its inhabitants. Of course, the subject 
matter is irrelevant to the comment. The systems mentioned 
above are facilities (environments) that extend the student’s 
programming experience. By way of contrast, Seidel and 
Kopstein’s system at HumRRO is a computer controlled 
learning system that also (incidentally) teaches COBOL 
programming. (2) The computer is used to control 
community or school activities by differentially routing 
individuals through various educational tasks (for example, 
practical work, reading, programmed instruction). This is a 
promising field, so far chiefly represented by Flanagan’s 
PLAN project. 43 

1.4. A Couple of Common Myths 

Even with these exclusions the systems listed in 1.2 provide 
sufficient evidence to destroy the common myth that CAI 
is wedded in principle to the rather tedious paradigm of a 
question and answer routine. It is not. 

Before going on, I would like to discredit another myth 
(mentioned in 1 .1) which does a great deal to hamper 
development in CAI. The myth in question is that the 
educational goal of the control system must be fully and 
formally specified. Of course, the goal of the controller 
itself must be well defined: but that does not mean that we 
have to know the “right” answer to each question. There need 
not even be a “right” answer. It is still perfectly possible to 
design a heuristic device that encourages learning. 

The following hoary and whimsical example demonstrates 
the point. Suppose you make utterances Xj, x 2 . . . (all of which 
are members of a set X) to a friend and that the friend replies 
with utterances, gestures or grimaces , y 2 . . . (selected from 
a set Y). A very simple minded control scheme amounts to 
rewarding certain of the x, y relations with a nod of approval, 
i.e. you choose a mapping 0 o : X->Y and reward all y such 
that y = 0 o (x). Here, the mapping 0 O is the crux of a fully 
specified educational goal. Equally well, however, you might 
choose to reward any consistent relationship, i.e. your friend 
could, unknown to you at the outset, select any 0, not your 
0 o and you would reward him provided his usage is regular 
thus encouraging him to learn some relation. Of course, you, 
as the controller, have a well-defined goal (mathematically it 
is of the form: maximize— dH(X, Y)/dt where H(X, Y) is the 

contingent entropy of the selections from X and Y but you do 
not have a fully specified education goal as you did in the 
first variant of the experiments. 

The principle involved is neither restricted to trivial 
systems nor to simple minded reinforcement procedures. For 
example, it is easy to set up an adaptively controlled system 
in which the student is provided with more items to classify 
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as he shows evidence of producing any self consistent and 
informative (efficient) scheme of classification (the student’s 
reaction is interesting in its own right; people who like to 
innovate find life in this libertarian teaching system 
exhilarating; those who do not, find it thoroughly 
disturbing). 44 There is no reason why the same ideas should 
not be. applied to much more complex teaching situations 
and a moment’s contemplation indicates that similar notions 
are implicit in the operation of any open ended 
conversational system. 
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i.5. The Evaluation of Control Methods 

The categories of control listed in 1.2 have psychological 
cogency; they correspond to recognizable situations or 
relations between the student and the machinery. They are 
also quite realistic. Given a skill or a body of subject matter 
to be taught, and given a description of the student 
population it is possible to come up .with estimates of the cost 
of each sort of control in terms of hardware and software. 
Further, a good deal is known about the behaviour and 
relative efficiency of systems Type 1,2,3, and 5 (when a 
skill and a student are specified).* 

Evaluation is possible because the control methods are 
specified relative to models for learning; hence the methods 
may be evaluated in respect to real systems with which the 
model in question is identifiable to form a theory of learning. 
The nature of these theories is taken up in Section 2. But we 
comment, at this juncture, that these theories are 
educationally relevant insofar as they treat subject matter 
organizations and modes of cognition that are 
characteristically in the human domain and with which all 
teachers are familiar. Hence they are not amongst the simplest 
sorts of learning theory which appeal to experimental 
phychologists because of their elegance and structural 
parsimony; the simplest theories are often (and perhaps 
rightly) discounted on the grounds that they have little 
relevance to education. The present batch of theories are 
proof against this criticism (and so are the system evaluations 
derived from them). However, both the theories and the 
evaluations can be attacked from a different quarter. 

The fact is, the theories are most readily testable when the 
corresponding learning models are identified with laboratory 
situations. There are several reasons why this is so; the models 
themselves are workable with respect to small (though, as 
above, representative) chunks of mental activity. At an 
experimental level, the effect of individual differences 
(between subjects) becomes embarrassingly large if the 
experiment is unduly prolonged (by “pure” psychology 
standards the experiments are fairly lengthy; even so they do 
not often last more than a day or two). Finally, there is a 
purely practical reason why the control systems have been 
tried out in miniature. Most of the work in this field has been 
conducted in laboratories which (until recently at any rate)** 
have had special purpose on line control gear rather than on 
line equipment controlled by a general purpose computer 
(as a half-facetious, half-serious aside, workers having “big” 
CAI interfaces seem to have been preoccupied with getting 
them going rather than finding out what they do). Under 
these circumstance , the cost of experimentation is appreciable 
aud it is necessary to choose experimental situations that are 
as small as possible. 

Because of all this, it is perfectly possible to maintain that 



the theories and consequently the control methods attached 
to them have been tested in conditions that are educationally 
picayune. That is a caveat which must qualify all of the 
comments in the next few paragraphs and it may or may not 
be viewed as a damning one. My personal conviction 
(supported by the argument in Section 2) is that the 
educational relevance of the theory and the test situation are 
of primary importance and that the time scale of the 
experiment and the precise character of the problem solving 
activity do not matter too much. 

The following notes are grossly generalized summaries of 
the main findings. 

If a skill or body of knowledge is unstructured, i.e. if 
students are unable to say how they learn it after careful 
interrogation, apart from a statement to the effect that 
they engage in practice, then a fixed training routine (a 
feed forward procedure) appears to be as good as any other. 
Acquisition is just a matter of repetition. Type 5 control is 
useful in maintaining the student’s motivation and may be 
used to check his perseverance. 

Rather few skills are completely unstructured. Presumably, 
the skills (if any) acquired by simple conditioning are of this 
sort. In the later stages of learning (but not at the outset) 
many perceptual motor skills are unstructured. The same 
comment may apply to the later stages of learning 
computational skills (mental anthmetic) and possibly (though 
probably not) to the later stages of language acquisition. Rote 
learning, if it really took place, would be a matter of 
repetition but, unless precautions are taken to prevent him 
doing so, the student structures almost any list or catalogue, 
however meaningless it seems to be. 

In contrast a skill (body of knowledge) may be structured 
in the sense that (I) it can be represented as an hierarchy of 
subskills or (II) a goal-subgoal hierarchy or (III) an hierarchy 
of TOTE units or (TV) an hierarchy of concepts or (V) if 
learning can be conceived as the elimination of a finite 
number of Error Factors. If the skill is structured (and the 
student can be persuaded to see it in this way) it becomes 
possible to talk about the existence of learning strategies. 

Suppose that one such strategy has been selected on logical 
oi psychological grounds as a good strategy. Now, a Type 1 
Feedback control system designed relative to this good 
strategy is more effective than either (1) a fixed (feed- 
forward) routine based upon the same strategy or (2) Free 
learning. The advantage becomes marginal as (a) the 
acquisition of each component of the skill (body of 
knowledge) approximates “one shot” learning or (b) the set 
of possible strategies are uniformly “good”. 

Consider a family of learning strategies derived from a 
single good strategy and a Type 2 control system based upon 
this strategy family. In general, the Type 2 system is more 



* The evaluation of Type 4 control depends ceiticaliy on the material 
Sometimes a game like mode of instruction is mandatory When a game 
is optional there is evidence to suggest that Type 4 control is a relatively 
effective method of instruction. 

** The picture in this country is as follows: Kay and Sime have 
recently set up a computerised installation at Sheffield. Annett and 
Duncan at Hull, have an interface linked to an Eliott 903 which is just 
becoming operational. Cook, Hartley and Sleeman have a system with 
multiple hardware (KDF9, Modula ^.1.) at Leeds. Gaines still relief 
chiefly upon special purpose equipment. So do I, though we are 
considering the economy of using a general purpose machine. It should 
be emphasised that the experimental requirements are fairly quirkish. 

A simple on line terminal is virtually useless for this purpose. 
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effective than the corresponding Type 1 system, if either 

(1) there is interference between the acquisition of one part 
of the skill (body of knowledge) and some other part of it or 

(2) the student needs to build up a learning set in order to 
secure competent performance (“interference” and “learning 
set” may be interpreted within any of the frameworks (I), 
(II), (III), (IV), (V)). 

Suppose there is a class of learning strategies which may 
be adopted by different students (or the same student on 
different occasions). It may not be possible to single out a 
strategy as “good”; for example, because some student uses 
each strategy effectively, i.e. we could only say “tliat strategy 
is good for a student with certain characteristics”. In this case 
a Type 3 conversational control system is more effective than 
either a Type 1 or a Type 2 system designed on the basis of 
an arbitrarily chosen good strategy and is very much more 
effective than free learning. The merit of Type 3 appears to 
be greatest when the influence of cognitive fixity (getting 
stuck with an inappropriate strategy or selection) is most 
obtrusive. As noted in 1 .2 Type 5 control may be used to 
achieve Type 3 control, either in the context of a group of 
otherwise independent individuals or in the context of a team. 

The majority of evaluations are expressed in terms of trials 
or time to reach a criterial performance with some check 
upon retention. A few are expressed in terms of uncertainty 
reduction. A more telling measure might be the time to reach 
a given level of understanding as indicated by the extent to 
which the knowledge in question is locked into a cognitive 
network. Tests for such a property have been devised but 
have not been extensively used. 



1.6. Control Methods as Educational Facilities or 
Educational Subroutines 
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There is little difficulty in isolating skills such as list learning, 
relation learning, series completion or concept attainment for 
which one of the control methods listed in 1.2 is better than 
the rest, and to determine the cost of instrumenting it. 

However, the majority of curricula— in mathematics, 
statistics, history, etc., call for the inculcation of several skills. 
This comment also applies outside the academic sphere; in the 
instruction of many complex tasks. Here, the picture is a 
good deal clearer since the subject matter is not so 
conventionally (perhaps arbitrarily) ordered. 

In COMCEN operator training, for example, it is possible 
to justify the use of all five control procedures (listed in 1.2) 
at various stages in the process. 45 This is true, even if w^ 
confine our attention to the code teleprinting skill which is 
one ingredient of a COMCEN operator’s repertoire. 

At the outset, there is a strong element of problemsolving. 
The student has to form concepts to do with different parts 
of the keyboard, and the acquisition of one demonstrably 
interferes with the acquisition of another. A similar situation 
occurs at a later stage, when the student is mastering the 
many to one relationships entailed by uppercase and lower- 
case operation. With the “difficulty” variables chosen as 
cueing and pacing, a Type 1 feedback control (Section 1 .2) is 
empirically superior to mere practice and, in certain 
circumstances (if the student’s behaviour goes outside a 
limited range), the Type 2 (adaptive) control is still better. 

In our own system, COMOPTS 1, the demand for Type 1 and 
Type 2 control is met by providing an overall feedback 
control teaching system in which the student is normally 



situated and an adaptive system into which he is routed for 
remedial practice. 

Up to 30 hours of conventional training (about 10 hours 
in the mechanized training system) there are several possible 
learning strategies. Since this is so, a conversational system 
(Type 3) is required (if only because we are unable to 
determine an appropriate teaching strategy on behavioural 
evidence alone). It is provided in a partially mechanized form, 
by a “programme” or “flowchart” in which the student 
substitutes values derived from the feedback trainer after 
each exercise block. Further substitutions are made to 
indicate the student’s preferences and to point out aspects of 
the skill that are causing peculiar difficulty. Following 
through the substituted flowchart, the student is directed 
either to alter parameters of the feedback machine, to 
rehearse the skill on the adaptive machine or to engage in 
other activities (which will, in the final version of the system, 
involve working through segments of a programmed text). 

After 30 hours of conventional training, the task is 
substantially homogeneous (from the student’s point of view) 
and the acquisition of further competence depends chiefly 
upon practice. Here, an open loop training system would be 
as good as any other (given only a minimum monitoring of 
performance). The main problem is to keep the student 
motivated as he grinds away. For this purpose, COMOPTS 1 
will make use of control methods Type 4 (simulation) and 
Type 5 (gaming), i.e. students will be periodically presented 
with realistic communication situations and periodically 
required to play partly competitive and partly co-operative 
games with one another via the teleprinter interface 
(augmented by a board that delivers commands and 
synchronizing signals). The frequency and the nature of these 
interventions are automatically scheduled as a function of 
performance which is measured during the communication 
act. These measurements provide sufficient information for 
monitoring purposes. At the moment, the monitoring is 
manual but it could be automated without difficulty. In a 
programme such as TASKTEACH (which is capable of 
teaching quasi intellectual skills, typically fault detection and 
maintenance), an instructor sets up the list which gives the 
system a structure akin to COMOPTS; within this framework 
the student is free to choose his own fate (conversational 
Type 3 control) or to invoke more restrictive procedures 
(resembling Type 1 and Type 2 control). 

Similar comments apply to PLATO, Kopstein and Seidel’s 
IMPACT and other large CAI systems. However, since these 
are generally used to instruct academic skills (statistics, 
computer programming and the like) the simple pattern is 
obscured. Statistics, for example, already has a conventional 
order imposed upon its instruction so that the plan for 
teaching the subject matter as a whole contains phases in each 
of which the control methods of 1 .2 are called into operation, 
generally with different parameter settings to achieve specific 
subgoals. In the context of a large CAI system the control 
methods of 1 .2 thus have the status of tutorial subroutines. 

It should be emphasized that the distinction between a 
tutorial subroutine and a full CAI system is made on the 
grounds of organization rather than size. To see this, notice 
that the Stanford System (Suppes and his colleagues) consists 
in a series of isolated subroutines which are called into 
operation at the discretion cf a teacher or supervisor. Thus 
the “drill and practice” mode, used for teaching children to 
carry out arithmetical operations, is a large but simply 
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structured feedback (Type 1) subroutine; the “tutorial” 
system employed for logic instruction (and other purposes) 
is a mixed conversational and adaptive subroutine (a hybrid 
of Type 2 and Type 3). The Stanford system has, 
nevertheless, massive provisions for recording and evaluating 
data and the system itself is one of the largest and most 
liberally interfaced* in existence. Of course, if the external 
supervisor uses the data to select the subroutines or to adjust 
their parameters then the system (including the supervisor) is 
a full CA1 system. Without the supervisor it is not. 



1.7. Evaluation of the System as a Whole 

It can be stated with some confidence that a full CAI system 
is no less effective than a system of programmed instruction 
or a system of classroom instruction. The more cautious 
commentators, seeking a general evaluation of the art, have 
gone no further than this and have still managed to justify 
CAI on economic grounds for higher education, special 
civilian and military training (where instructional costs are 
high in any case), and for many industrial and governmental 
purposes. When relatively inexpensive systems (such as 
Bitzer’s) come into operation the cost of CAI will fall from 
about $2.4 per student hour to about $0.35 per student hour 
due, in no small measure, to large-scale production techniques 
and hardware innovation; this change will place CAI within 
the economic compass of the school system. 

But all of these estimates, predicated on “CAI is no worse 
than . . .”, may be unduly pessimistic. In view of the 
information available about the basic control methods 
(educational subroutines), it is not unreasonable to suppose 
that CAI may be “a great deal better than . . .” other 
techniques and, if so, any cost benefit argument in its favour 
should be more readily acceptable. 

Unfortunately, the evaluation of a total CAI system is 
beset by a number of difficulties. Some of these have already 
been mentioned; for example, the problem of coping with 
individual differences. However, there are five obstacles 
If which deserve special attention. 

H (1) Providing it has been competently designed, any CAI 

system whatever is bound to give a saving in time and effort. 
!«• For example, we can claim that COMOPTS will train 
1 1 teleprinter students (up to 3 or 4 week level) in half the time 

"* taken by a conventional training routine. But we have a 

relatively vague idea about how much of this effect is due to 
|| the tutorial action of the system. Some of the benefit stems 
I* from an incidental rationalization of improvident training 
techniques; in specifying any CAI system an efficient 

E planning procedure is forced upon the designer. In particular, 
the CAI system is far more flexible and individualized and the 
greatest organizational advantage is gained when it is possible 

E to get rid of the class attendance concept (so that the system 
can be used optimally with respect to the pupils). The Army 
Operational Research Group 46 in a much earlier study, 
predicted a similar enhancement in training'efficiency (for 
jT morse training) on these grounds alone; without reference 
Jg to CAI as such. 

Needless to say, the same comments apply with even 

I n greater force to CAI systems used in the academic sphere. 

The enhancement due to organizing instructions and avoiding 
1 . time wastage is particularly great if, with Eide , 47 we count the 

I I Dj , * For example, the student receives spoken messages in addition to 

visually displayed material. 



students’ time as one of the most costly educational resources 
(there is a very strong case for doing so at the national level). 

(2) Any competent CAI system will work because of the 
Hawthorne effect. It is difficult to discount the influence of 
novelty in an experimental situation . Desirable though the 
improvement may be, it is not directly relevant to the tutorial 
influence of CAI. 

(3) Whereas it is easy enough to find standard conditions 
for controlling small-scale experiments (free learning or a 
fixed training routine) it is not so easy to select a control 
condition for a full CAI system (glibly CAI performance is 
compared with the performance of “conventional teaching”, 
but what is “conventional teaching”? It is fairly well defined 
for perceptual motor skills but not for academic skills). 

(4) Individual differences are not only large (a point made 
already in 1 .5) but specific to the motivational variables that 
appear in the fully-built CAI system. 

(5) The final and fundamental difficulty is that no one 
really knows what to measure (as evidenced by the fact that 
vast amounts of latency and response data have been collected 
on tapes, though very little of it has been used for evaluation). 
Just as the functional evaluation of an educational subroutine 
calls for a learning theory (which, amongst other things, tells 
the experimenter what to measure) so the evaluation of an 
entire CAI system calls for an educational theory. At the 
moment no adequate theory exists, though several relevant 
developments are noted in Section 2. 

Thus, at the moment, we are in the position of aprogrammer 
equipped with a collection of working and evaluated 
subroutines who is anxious to evaluate a programme (the full 
CAI system) constructed from these components. It would 
be naive to suppose that the evaluation of the whole (in the 
educational case) is the sum of the evaluations of the parts. 

In some instances, useful limits can be set up in terms of 
subroutine evaluation but a general appraisal of CAI seems to 
rest, as suggested above, upon an educational theory. We 
return to this point in Section 2.8. 

1.8. Critical Statement 

It is true that computers have often been used (underpressure 
to use the things somehow) in an unimaginative, narrow or 
downright stupid fashion. But this should not lead to a 
condemnation of the entire field: computer control of 
learning has great and ‘diverse potentialities, some of which 
have already been realized. 

The general development of computer control does, 
however, entail paying more attention to psychology and 
logic than it is currently fashionable to do. 

To take just one aspect* of the psychological point, most 
of the varieties of control listed in 1 .2 call for an adequate 
interface (which is certainly not a teletypewriter) and an 
adequate language. These questions are really in the province 
of other authors at this symposium; I only wish to make two 
points which illustrate a broad contention: (1) any 
conversational system must have an interface and a language 
which allows the student to state what he intends to do, not 
only as a plan for action but also as a plan for learning; (2) by 
the same token, the interface and language must allow the 
student to evaluate prescriptions and descriptions either to 
do with the task itself or with the process of learning it. One 
of the most useful evaluations is a subjective uncertainty 

* We return to it in the section on learning models. 
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► measure, preferably obtained with a Shuford 48 type of 

scoi ng function as mechanized by Baker. 49 * 

. Ihese facilities are mandatory adjuncts to quite simple 

i studies of teaching and it is disheartening to find elaborate 

i computerized systems which lack them altogether. Needless 

*. to say, the systems cited in this paper, especially Seidel and 

< Kopstein’s system, do have such facilities. So do many 

I European and Russian systems which are excluded from the 

? survey. 

' On the issue of logic, there is an understandable tendency 

l to see things through the eyes of computation technology. 

* Logic is parochially (and naively) identified with 

f propositional and first order predicate calculus. But, insofar 

| as a logic of teaching is a logic of conversation, this is an 

l insufficient framework. It is at least necessary to synthesize 

1 a scheme from the framework as it stands and (a) a logic of 

) relations (the higher order predicates); (b) a logic of 

r commands (Rescher 50 and von Wright 51 ); (c) a logic of 

!. parallel (rather than sequential) organizations, for example, 

; using Holt’s scheme; 52 (d) a context logic (in the sense of 

i Kotely 53 ); (e) a logic of distinctions and form (Spenser 

Brown 54 ) and (f) a logic of ga. ;es and metagames in the sense 
\ of Howard. 55 Some efforts have been made in tb s ■ direction, 

| notably by the social anthropologists who face much the 

r same problem as “learning system analysts”. It is plain silly 

I to pretend the problem does not exist, to give up in despair 

I or to bludgeon away with techniques that serve well enough 

\ for describing mechanical computation (as carried out by 

l present-day computers) but are unfitted to adumbrate the 

i dynamics of mentation. 



2.2. The Scope of the Models 

We have talked of learning theories and student models as 
though these things were neat packages. In fact, there are 
several separate representations which form part of any model J 
and which are given varying amounts of emphasis. 

(1) A representation of the subject matter to be instructed. 

(2) A representation of the educational goal. In the simplest 
case this is a statement of the looked for terminal behaviour 

but frequently there are cognitive components in the L 

specification. j 

(3) A representation of the initial state of the student -~i 

when he enters the teaching system i.e. of his current level of j 
competence, his attitudes and possibly his personality. 4 ^ 

(4) A representation of the current state of the student __ ' ■ 

(3) above providing a statement of the “initial conditions”) [ 

Generally, this part of the model is far more detailed and is i J 

mainly restricted to a representation of the student 

competence and his learning strategies (if attitudes, fl 

personality etc. appear in the model at all they are assumed j 

to remain unchanged by learning). 

(5) A representation of the teaching system including its . 

teaching strategies. j 

All of these representations are present (perhaps in an 
embryonic form) within competent (designer’s) model of the 
student and his instruction. Models in the system are set 
within the framework afforded by ( 1), (2) and (3) but ; _ 

(insofar internal model is changed as a function of 
experience, die internal model itself is restricted to y 

components (4) and (5) . 



2. MODELS FOR LEARNING AND TEACHING 
2.1. Overview 

Any controller is designed according to a model which can 
be interpreted and identified as a theory of the controlled 
process. In the case of teaching (the control of learning) this 
theory is a learning theory and it represents either the student 
alone or the student coupled to a specific teaching system. 

We emphasize that the underlying model is the designer’s 
model of the process; it is he, the designer, who interprets it 
as a theory. Furthermore, it is he who carries out experiments 
to verify the theory and it is he who, if necessary, alters the 
model to fit the facts. 

In contrast, models may also be used inside systems. Here 
the teaching device is provided with an internal model which 
it is built to interpret as a theory of the student with respect 
to the educational goal. It uses the model to determine its 
next move, it performs experiments to validate the model, 
and, if necessary, it modifies the model. In other words the 
system leams about the student aind builds up an internal 
representation of the student state. The distinction between 
(designer’s) models of the system and (system) models in the 
system is non trivial when the system acts in this fashion. All 
conversational systems and most adaptive systems do so. 

(It is of interest to note that the system might start from 
scratch and build up a student model de novo. In practice this 
process would take far too long and even if it could be 
instrumented would clearly constitute an uneconomic design.) 



O * Other techniques, such as a mechanised version of the Kelly grid, 
: ERJ could be introduced with advantage. 



2.3. Some Types of Learning Theory 

Confining our attention to components (3) and (4) of 2.2, 
there are several types of learning model available to the 
designer of a CAI system. (Many of these are only 
interpretable on the “subroutine” scale considered in 
Section 1.2). 

In order to appreciate the state and potentialities of 
CAI it is necessary to review the major types of learning 
theory and to comment upon their ranges of application. 

As a prefatory remark the various theories are often 
complementary to one another; the differences are mainly 
philosophical. At a pragmatic level the recommended teach' lg 
strategies tend to be quite similar (as they should be! the 
theories are theories of the same system!). Nevertheless, one 
point of view may be very much more useful than the rest 
when it comes to modelling a particular situation. 

(1) Behaviouristic Models. The operant conditioning 
approach of Skinner was largely responsible for “linear” 
small step programming (a teaching paradigm still at the roots 
of many CAI systems). More liberal concepts of behaviour 
shaping have led to the “Mathetics” type of programme 56 
(which consists in large frames, eliciting many covert and 
overt reponses, each one resembling a well planned 
advertisement).* The S-R-operations (apart from basic 
conditioning and association) are chaining, discrimation and 
generalization and these operations are evoked in sequence in 



* In terms of this paper, both Skinnerian and “Mathetical” techniques 
are “feedforward” techniques. The student received a reinforcing 
feedback but feedback signals are not employed to modify the sequence 
or type of instruction. 
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order to produce (by hypothesis) such a juxtaposition of 
associative connections that the criterion behaviour can be 
achieved. 

The. main attraction of this theory is its simplicity, 
atomicity and quasi physical calibre. Its main disadvantages 
are (a) that the structuring of the task (representations (1), 

(2) and (3) of 2.2) is outside the model and is catered for 
informally (the matter is left to the designer’s discretion). 
(b)The representation of such entities as concepts, rules and 
plans ((4) and (5) of 2.2) is exceptionally cumbersome and 
may even be impossible. 



(2) Stochastic learning theories, especially the stimulus 
sampling theory of Estes and his colleaguec. 57-58 Stochastic 
models are admirable predictors of the statistical properties 
of simple behaviour, for example, of response probabilities 
(strictly, these are properties of an ensemble of students but 
they may be interpreted as properties of processes which are 
ubiquitous and thus represented in any student). Some typical 
and interesting applications to teaching are Dear and 
Atkinson’s 59 model for concept learning and Matheson’s^ 0 use 
of a stimulus sampling model in combination with the 
technique of dynamic programming to prescribe optimal 
teaching strategies for certain (more or less repetitions) tasks. 
The control and evaluation procedures in Smallwood’s 61 CAI 
system are based, primarily, upon stochastic learning theory 
and this scheme, though not so highly developed as 
Matheson’s* is applicable to a much wider range of subject 
matter. Hence the stochastic models have much to 
recommend them. Their disadvantages are those voiced in 
connection with the behavioural models and, in addition, the 
limited tutorial relevance of the statistical properties they 
predict so well. 

(3) Error Factor Theory. Within this (more or less 
behaviouristic theory) learning to do something is regarded as 
the elimination of the influence of Error Factors which 
prevent it being done and which are responsible for classes of 
mistaken responses. Error Factor Theory 62 is particularly well 
suited to designing feedback and adaptive control systems for 
perceptual motor and problem solving skills. It has the great 
advantage of imaging the response process as potentially 
multidimensional and the vector of difficulty variables 
(Section 1 .2) can be placed in direct correspondence with the 
set of Error Factors. The forms of learning predicted by Error 
Factor theory and simple reinforcement theory are somewhat 
different but the recommended training strategies are much 
the same (Error Factor theory yielding consonant but more 
detailed recommendations). 



(4) Functional Theories. Enough is known about certain of 
the subsystems involved in learning to model them 
meaningfully as mechanisms (rather than “Black Boxes”).* 
This is especially true of the functionally demarcated 
subsystems “Immediate” (short term) “Working” 
(Intermediate) and Long Term memory, of which the first 
two are fairly well investigated. 



* In Ashby’s sense; modelling the input output characteristics or 
transfer function of a system. 



The characteristics and limitations of immediate memory 
are familiar from work in pure experimental psychology; 
clearly these limitations should be respected in the design of 
any CAI system (as in the design of any ether man machine 
interface). The working memory system is less well 
documented but is especially germane to teaching since it is 
in this system that skills and bodies of knowledge are 
integrated into structures and it is here that interference and 
positive transfer of training take place. 

There are various models for the system, but Atkinson 
and Shiffrin’s 64 is well validated and is stated in a form 
applicable to CAL The working memory is akin to a 
programme embodied in a working storage which serves as 
a general purpose computer that may be programmed in 
several ways. Each way is an information control procedure; 
for example, one procedure is a rehearsal buffer, another is 
a sort of push down list processor, another is a procedure 
for transfer to long term memory. Fiegenbaum’s EPAM 
programme, 6S_66 a computer simulation of working memory 
viewed as a discrimination and association network, is a 
further model of this type and is also well validated. Given a 
theory based on one of these models, it is possible to 
recommend very detailed teaching strategies. Several of these 
have been tested as part and parcel of the experimental effort 
connected with the modelling. It should be noted that the 
teaching strategies derived from functional theories are 
generally compatible with those derived from behavioural 
theories. But the functionally derived strategies are far more 
sophisticated. 

(5) Objective Informational Theories (Wattanabe, 67 
von Foerster, 73 Pask 68J72 ). The word “information” 
is used in its technical sense (selective information, measures 
of information) «oid learning is viewed as a reduction in 
relevant uncertainty which is influenced by such variables as 
the student’s information loading and the informational 
redundancy of the materials presented to him. “Objective” 
implies that the informations, uncertainties, etc., in question 
are those of an outside observer, not of the student. On the 
one hand such objectively measured uncertainties are clearly 
related to statistical indices such as response probabilities 
(regard stimuli as signs selected from an input alphabet and 
responses as output selections). Thus the informational 
theories of learning are. related to the behavioural theories. 

On the other hand, the information measures, though obtained 
over sets of behavioural alternatives, may be held to estimate 
the student’s uncertainty. Insofar as they do so a bridge is 
established between behavioural theories and subjective 
information theories and via these between behavioural 
theories and cognitive theories of learning. 



(6) Subjective Information Theories. The information 
measures are computed from subjective estimates of degree 
of belief (or doubt) obtained by the methods of Shuford, 
Baker and others.* These workers have developed a learning 



* Essentially in a game like situation where the student is assigned a 
score dependent upon his numerically asserted degree of belief in 1 

several alternatives. His assertion is constrained to secure “probability" 
numbers which sum to one over all of the alternatives. The scoring 
function employed is such that, over a sequence of trials, the student’s 
mathematical expectation of score is maximised if, and only if, his 1 

asserted degree o f belief equals his real degree of belief. , 






theory which is comparable with an objective theory and have 
used it to prescribe various teaching strategies. 

(7) Structural Theories . The developmental psychologies of 
Piaget and Luna* are structural theories insofar as they 
postulate certain mental structures which dominate the psyche 
at different ages. At the level of individual learning (rather 
than development) there are several theories of a like kind; 
Gagne, 75-76 for example sees the acquisition of a skill as the 
assembly of an hierarchical structure of capabilities. Learning 
occurs when a pair or more of subordinate capabilities are 
wedded together by an instruction that combines them in 
some way; but the instruction will only be effective //the 
requisite subordinate capabilities are already established. For 
dealing with a body of knowledge (rather than a skill) the 
“capabilities” are replaced by “concepts” (given a behavioural 
emphasis a concept is a capability to classify and respond). 

(8) Organizational and Cybernetic Theories. These are 
structural theories in which the basic units built into an 
hierarchical structure are goal directed, problem solving or 
control systems (either contained within the organisation or 
acting partly through the environment. The theory has a 
cognitive aspect; since “aiming for a goal” or “solving a 
problem” explicitly involves a state of knowing. Miller, 
Gallanter and Pribram 77 proposed a TOTE (test operate test 
exit) unit as the goal directed building block; essentially this 
is a piece of programme which corresponds to a contingent of 
IF, THEN, ELSE, statement. A plan or a concept is conceived 
as a nested structure of TOTE units. 

I have pointed out that in order to build up or modify 
such an hierarchy (i.e. in order to talk about learning) it is 
necessary to invoke an independent hierarchy of control (or 
problem solving of problem solving). Because of this any 
Cybernetic learning model is heterarchical l 78-79 ' 



(9) Programmatic Cognitive Theories. In general these turn 
out to be Cybernetic theories although the basic units are 
specified simply as information processing programmes, the 
execution of which may be expected to produce a state of 
knowing and the consciousness of a plan, hypothesis, 
concept or strategy.* All artificial intelligence models which 
purport to simulate human learning belong to this class. Since 
the models are very complex and varied, it is impossible to 
convey the gist of them in a few words but the subject has 
been reviewed in another paper. 80 Bennett and Hodges 
“Structural Communication” or “Systematics” type of 
programming is based on a cognitive modelf 1-82 informally 
so at the moment, but a formal treatment is clearly possible. 

It is also quite clear that although systematics programmes 
can be administered by textual methods, they are really fitted 
to CAI administration and have the status of CAI systems. 

(10) Other Cognitive Theories. Several perfectly respectable 
cognitive theories do not fit easily into the currently popular 
programmatic framework. Two good examples are Festinger’s 83 
cognitive dissonance theory and Kelly’s personal construct 
theory. 84 Both of them have potential value in connection 



* Unless the entity is specifically tagged as a reflexive or autonomous 
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with certain aspects of learning and teaching (notice for 
instance the close relation' between the “cognitive fixity” of 
1.2 and “cognitive dissonance”). 

(1 1) Statistical Cognitive Theories. Guildford 85 and 
Bunderson’s 86 factor analytic studies of learning showed that 
different mental processes (aptitudes) are important at . 
different stages in learning the same task. Clearly, these 
“processes” have a statistical calibre akin to the response 
“processes” in a stochastic model apart from the fact that 
they are cognitive rather than behavioural entities. A body of 
hypotheses frames in terms of there variables can be usefully 
interpreted as a theory (such a construct is used in the 
IMPACT model). 



2.4. Models and Theories of Learning 
The learning theories of 2.3 are concerned with the 
components or representations (2) and (4) of Section 2.2 
(namely of the student’s initial state and his current state). 

Given a specific task or body of knowledge it is fairly easy to ] j 
adjoin a representation of the subject matter (component (1) 
of 2.2) and of the educational goal (component (2) of Section r 
2.2) to yield a descriptive and predictive (or explanatory) 
theory of learning the task in question. Clearly, the 
representations employed for this purpose must be compatible 
with the type of theory chosen and it is almost platitudinous • 

to remark that the choice of a learning theory will depend L . 

upon the subject matter and the educational goal that an 
investigator has in mind. For example, the behavioural j = 

theories are simple and thus (in one sense) preferable. But it I 
is difficult if not impossible to set up a subject matter ; 

representation which is compatible with such a theory if the . - 

subject matter entails plans, concepts and the like, in anything j i 

more than a naive fashion.* In general, it is necessary to j 

reach a compromise between theoretical simplicity and ’ 

realism of subject matter representation and the structural [ \ 

and Cybernetic theories are peculiarly useful in this respect. !_ j 

Although the models of Section 2.3 are chiefly descriptive \ 

(and predictive) models, they can all be used in a prescriptive : i 

fashion** to yield recommendations for teaching; either a set I ' 

of teaching strategies or a teaching system. Thus, we enquire ~~ j 

what operations would be necessary to achieve the educational jj 
goal (with respect to a given body of subject matter and a | j • 

given student model) and specify a teaching model capable L I 

of performing these operations. The requirement may, of 
course, be more or less refined; for example, we might j ; 

demand a set of operations to achieve the goal as fast as 
possible or set of operations that lead to a generalized rather 
than a specialized competence. But, in any case, the teaching 
model constitutes component (5) of Section 2.2 and, when , t ] 

* This is a personal bias. For instance, I do believe it is naive to " 

regard a concept as no more than a class of stimuli that evoke some 
common response. ■ 

**The distinction between the descriptive and prescriptive use of : j 9 

models was originally drawn by Kopstein . It ha* been employed • -- i M 

extensively by Stolurow and myself. Stolurow also consider models m 

that have little predictive power. Thus, my usage is somewhat different. ,9 
All of the models cited in this paper, for example, have predictive or j j « 
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(12) Theories of representation and learning type, notably 
Bruner’s 87 theory of ikonic, mnemonic and automatic 
representation and the linguistic theories of Vygotsky, 88 
Luria and their pupils. 





coupled to the student model in the context of the task, it 
provides a learning and teaching model of which 
representations (1), (2), (3), (4) and (5) of 2.2 are constituent 
parts. 

The learning and teaching model may be a purely symbolic, 
mathematical affair. More often, however, it is convenient to 
embody the model in a special purpose artifact or a computer 
simulation, when the student part and the teacher part are 
explicitly distinguished as separate subsystems. It should be 
noted that the mode of interaction between the student part 
and the teacher part of the model depends upon the type of 
learning theory that was originally selected. For example, if 
the base theory is behavioural, then the “language” whereby 
the student part and the teacher part communicate is able to 
accommodate stimulus signs, response signs and signs for 
knowledge of results or reinforcements; if it is structural, then 
certain expressions designate descriptions and commands or 
instructions; if it is a subjective informational theory, then 
the student part produces evaluations (of degrees of belief) 
and if it is Cybernetic or cognitive then the language must be 
an altogether richer approximation to natural language. 

The learning model can sometimes be validated directly 
and used to prescribe a teaching procedure. However, it is 
also possible to employ the entire learning and teaching model 
as the experimental tool in a very efficient fashion.* To do so, 
the joint model is set in motion (or “run”) to generate 
predictions; as a result of which interaction or “discourse” 
takes place between the parts. The predictions are of the 
form “The students learning process will be like X if the 
teaching strategy is Y”. Initial values of the “student” 
parameters are assumed and the parameter values in the 
teacher part of the model are adjusted in order to secure the 
requisite tutorial functions. This is a period of abstract 
experimentation or simulation during which the teacher 
component is refined to yield'a final prescription for a 
teaching system. At this stage, the joint model is interpreted 
as a theory of teaching', the student part is identified with a 
real life student; the teacher part with a CAI system (or a 
CAI subroutine) the interaction between the two with the 
student teacher discourse and the subject matter and the 
educational goal with their correlates in reality. The real 
teaching machine is now used to instruct members of a group 
of students and data gleaned from this real experimentation 
is used to modify the student model before the entire cycle 
of operations is repeated. 

2.5. Models in the Teaching System 

This completes the story so far as simple feedback systems 
(Type 1, Section 1.2) are concerned. For example, Matheson’s 
dynamic programming model for a teacher leads to a set of 
well defined teaching strategies that are optimal provided that 
the learning process is adequately described by a stimulus 
sampling model. These strategies have been successfully 
embodied in real teaching machinery. 

On the other hand, the prescription for a teaching model 
may be a “learning model” (a model that learns about the 
student, a model inside the system in the sense of Section 2.1). 
This may occur for either of two reasons: (1) the characteristics 
of the student, are unpredictably variable (either from 

* Since many of the learning models are easily validated in on line 
controlled experiments (notably Atkinsons and my own) the distinction 
between the learning teaching procedure and ordinary experimentation 



individual to individual or from time to time) or (2) the 
student must be allowed a free choice (say of learning 
strategy) if his learning is to be optimized. 

If the teaching model is, itself, a learning model then the 
abstract experimentation phase, mooted in 2.4 assumes the 
status of an on line process and the resulting CAI system 
becomes a learning system. All adaptive and conversational 
CAI systems are of this sort. 

Since these CAI systems unequivocally do something that 
cannot be done by a bit of promgrammed instruction, they 
deserve further attention and will be briefly discussed in 
Section 2.6 (at the level of CAI subroutines) and in Section 
2.7 (at the level of full CAI systems). 

2.6. Case Hisiories of Learning and Teaching Systems 

To illustrate these concepts, I shall briefly describe and 
comment upon a series of learning and teaching models for 
which I have been responsible in one way or another. All the 
systems in this series are addressed to “rule application” which 
is a skill required in the performance of many intellectual and 
perceptual motor tasks. (The student is given a rule or set of 
rules relating stimulus configurations to response alternatives; 
the rules may be simple or complex. At any trial, the student 
receives a stimulus configuration together with information 
that directly or indirectly indicates tire appropriate rule. He 
is required to produce a legal response as soon as possible. All 
in all, “rule application” is a tractably simple paradigm for 
“problem solving”.) 

The first learning and teaching model (EUCRATES) was 
fabricated in the late 1950’s before general purpose computers 
were readily available; hence, it was embodied in a special 
purpose hybrid computer. It is, incidentally, by far the most 
complex (though the least conceptually sophisticated) model 
in the series and even today, its computer simulation would 
be a sizeable undertaking. The next case is a simple minded 
informational-cum Cybernetic learning model which (like 
EUCRATES) was used to design adaptive teaching systems 
with a single teaching strategy. The model is computer 
simulated since the work in question was started about 
8 years ago. The last case is a recent and moderately 
sophisticated Cybernetic learning model (again, computer 
simulated) used to design conversational teaching systems 
based on several learning strategies. These teaching 
systems have been tested on real life subjects (Pask and 
Lewis, 8 * -90 Pask, 91 Pask and Scott 92 ). 

(1) The EUCRATES system (Bailey, McKinnon, Wood 
and Pask 93-94 ) has two functions (a) it is a learning and 
teaching model embodied in a special purpose hybrid 
computer and (b) it can be used directly as a teaching system. 
The main object of building it was to provide a facility for 
designing, evaluating, and testing simple adaptive teaching 

EUCRATES consists of a console for student interaction 
a learning machine and a teaching machine, both with 
variable parameters. The teaching machine is either coupled 
to the learner, when the entire system acts as a joint learning 
teaching model ((a) above) or via the console to a real life 
student; the teaching application ((b) above). 

The learning model is based upon abehaviouristic or Error 
Factor theory insofar as the rule or rules are never stated 
symbolically to the learner and insofar as the rule becomes 
internally represented as a set of stimulus response associations 
embedded in the association storage matrix of a conditional 



reinforcement probability device. However, the behaviouristic 
paradigm is liberalized in the following ways (a) the 
EUCRATES learner has a primitive attention directing 
mechanism, impelling it to focus upon submatrices of the 
storage matrix, (b) It can generate stimuli internally, even if 
no stimulus is presented, (c) It has curiosity (or a drive to 
learn about something); hence it does generate its own stimuli 
if none are provided, (d) It has a simple minded anticipation 
or expectation mechanism, (e) Given a stimulus (internal or 
external) a column of the storage matrix is selected; its entries 
are presented as a biasing vector to a set of response 
processes which compete to produce a response; the 
competitive activity is inhibited by cueing signals (from the 
teacher) indicating that a certain subset of responses are not 
correct (i.e. signals eliminating the influence of Error 
Factors). 

The EUCRATES teacher contains a representation of the 
rule or rules and comparators for providing knowledge of 
results or reinforcement signals. Apart from that it 
approximates the stochastic inverse of the learner. It consists 
of a variable probability stimulus generator (stimuli 
eliciting correct responses being selected with ihe lowest 
relative frequency). The output of this device picks out 
columns in an associative storage matrix. The resulting vector 
biases a cueing process analogous to the response process in 
the learner. Just as the learner generates a response with a 
latency and form dependent upon the currently ongoing 
activity, so the teacher generates a*set of variably delayed 
cueing signals that differentially inhibit the learners response 
process. The entries in the learners associative storage matrix 
(and certain other stored values) depend upon contingent 
reinforcement from the teacher. The corresponding entries in 
the teacher are built up as a function of the learner's success 
and, in aggregate constitute a developing model for the 
learner. Further, the teacher has the overall goal of 
withdrawing (delaying) cue information so that none is 
delivered and presenting the stimuli equi-probably in such a 
way that the learner is performing at the required level of 
proficiency (rectitude and latency) when these conditions 
prevail. Hence the teacher is an adaptive teacher. 

Typically, the learner is set up with sensibly chosen 
parameter values (change of associative linkage as a function 
of reinforcement, decay of associative entries, degree of 
inhibition). The system is run as a learning and teaching model 
and the teacher parameters are adjusted to approximate 
optimal convergence (abstract experimentation). The 
parametrically adjusted teacher is next used ai an adaptive 
machine to teach a real life student (real experiment) as a 
result cf which the learner parameters may be further 
adjusted and the cycle repeated. 

(2) In a Cybernetic model (Pask, Lewis, Mallen et a/ 94 " 96 ) 
the student is conceived as an interpreter rather than the 
reactive mechanism imaged by the behaviouristic theory. 

He interprets stimuli as posing problems within a given task 
description, he sets about solving them and he experiences 
uncertainty about how to solve them. 

At a macroscopic (molar) level, the novice is overloaded 
by a sequence of real life configurations in the sense that the 
uncertainty engendered by the problems they pose exceeds a 
threshold. Further, it is hypothesized (in the student model) 
that learning is most efficient if the student's uncertainty is 
held within limits (if he is maximally loaded but not 
overloaded). In order to learn at all, the student is furnished 



with a description of the skill within which problems can be 
decomposed into their constituent subpiobiems and he is 
able to focus his attention upon any class of subproblems. 
Hence the student is able to control his maximum uncertainty 
(determined by the complexity of the subproblems that 
occupy his attention). By hypothesis, he adopts a learning 
(attention directing) strategy in which thef maximum 
uncertainty is controlled as a function of actual uncertainty 
in order to satisfy the limits mentioned a moment ago. 
Concisely, the student acts as a self-organizing system in ihe 
sense of Von Foerster. This general view of human learning is 
supported by experimental data. 

At the microscopic (molecular) level, the same learning 
process is described by a computer simulated model which 
represents the construction, reproduction, and generalization 
of an hierarchy of TOTE units in an intermediate (working) 
memory system which is subject to interference. The process 
is controlled by an executive which assigns effort to the 
various levels of internal and external problem solving 
activity. Many of the more detailed predictions from this 
model are also confirmed by empirical data. 

Used prescriptively, the microscopic and the macroscopic 
models lead to the same recommendations. The teaching 
system should have a single strategy; the learning strategy 
already mentioned. It should act in a partially co-operative 
fashion (a) by sampling the students competence or level of 
relevant uncertainty; (b) by “partially solving the problems 
that are presented”, to a degree determined by the current 
competence level (equivalently the teaching system 
co-operates “by selecting subproblems of a difficulty 
dependent upon the student proficiency). The parameters 
involved in (a) and (b) are chosen to satisfy the limits mooted 
in connection with the student model. 

Insofar as these limits can be determined for all students 
(by experiment), the teaching system is a simple feedback 
device (Type 1 of Section 1 .2). If the limits are variable, as 
they are for some tasks, then the system must determine the 
limits appropriate to an individual by an adaptive process (it 
is a system of Type 2 in Section 1 .2). 

The learning and teaching mode* is a computer simulation 
cf the (microscopic) student model interacting with the 
(adaptive) teaching model. Sensible parameter values are 
assigned to the student part and the model is run (abstract 
experimentation) to determine parameters for the teaching 
part. A physical systfem isomorphic with the teaching part of 
the model is now used to instruct a real life student (real 
experiment) and the results from a group of experiments 
determine what (if any) parameters in the student model 
need adjusting before the cycle is repeated (in fact, one 
repetition is usually sufficient to yield the specification of an 
efficient teaching system). 

(3) The final model in the series 97-98 has greater cognitive 
realism and is based upon a class of learning strategies, rather 
than a single strategy. 

Experiments were carried out in a special mechanized 
system that allowed the’student to output his learning 
strategy* as an objectively measurable stretch of attention 
directing behaviour. Given a task description compatible with 
many learning strategies, real students, in fact, employ three 
or four different ones. However, in free learning conditions, 
students are liable to adopt and stick to an inappropriate 

* Together with his degree of doubt about where to diiect his 
attention and his doubt about how to solve the problems. 



strategy; inappropriate in the sense that its execution is 
hampered by constraints of the sort that are imaged in the 
microscopic student model. All but one of the learning 
strategies are logically defensible but effective learning takes 
place when the chosen strategy is matched to the data 
processing characteristics of the individual who chooses it. 

The student part of the learning and teaching model is a 
computer simulation that extends the “microscopic” model 
(cited above) to accommodate several learning strategies; 
indeed, the model has (a currently rather primitive) process 
for generating strategies. Used prescriptively, this model 
determines a conversational teaching system which gives the 
student model a maximum freedom of choice provided 
(a) that a learning strategy is chosen and (b) that this could 
be a “matched” strategy. Briefly, it engages the student in 
discourse, commits him to objectives, offers him advice about 
his own internal state (of which the real student is commonly 
unaware) and guides him through a programme that is a 
compromise between the strategy the student would have 
selected alone and the strategy he ought to accept on logical 
grounds. 

Conversational teaching systems (isomorphic with the 
teaching part of the learning and teaching model) have been 
realized and used to instruct re d life students. They have the 
effect of enhancing the performance of any student to the 
level achieved by those few students who attain excellence in 
the free learning situation. It is of some philosophical interest 
to observe that a conversational teaching system is in a very 
real sense an extension of the student’s own faculties. 



2.7. Case Histories-Some Fully Specified Systems of CAI 



All currently instrumented CAI systems consist in tutorial 
subroutines that are glued together by the subject matter 
structure and the educational goal (by components (1) and 

(2) in the discussion of 2.2; the names for it vary, sometimes 
the subject matter structure is called a “concept ordering”, 
sometimes a “job analysis” and so on. We shall comment on 
the status of the subject matter structure in Section 2.8. For 
the moment, in order to describe a few CAI systems, let us 
assume that it exists. 

(1) The TASKTEACH system (Ridgeway and Towne 16 ) is 
a CAI procedure for teaching “serial action” taks such as 
problem solving. For any particular task certain lists and 
other information structures are set up by the teacher (task 
programmer). As a result the system makes available (a) an 
overall map or plan of the task which is open to the students’ 
inspection;* (b) a “smorgarsbord” (the author’s term) of 
educational facilities and “supports” to which the student is 
allowed conditional access. In the case of fault detection 
training, for example, these will include explanatory routines, 
test equipment, signal and state simulations. Clearly, the 
subject matter structure is embodied in (a) and (b). 



o 



(2) The TASKTEACH programme is conversational insofar 
as it allows the student to choose his own learning strategy 
(within the limits set by the overall map) and insofar as it 
provides him with evaluation data when he uses the 
educational facilities. It is a great advance upon straight' 
forward programmes which present an orderly sequence of 
frames. All the same, it leaves most of t he decisions up to the 

* The designers make the extremely important point that this, and 
certain other TASKTEACH facilities, encourage the student to learn 
low to plan his own learning. 



student and there is no explicit sense in which it secures a 
compromise between the student’s desires and its own rules. 
From a scrutiny of the single available paper, the learning and 
teaching model behind this CAI system is partly behavioural, 
partly cognitive and partly structural, the design is not 
committed to one particular theory. 

(3) The PLATO system (Bitzer and his colleagues at the 
University of Illinois). This is an impressive CAI system which 
(like TASKTEACH) permits a great deal of student control. 
However, it does so within a much more liberal format 
(PLATO has been used to teach nearly everything; network 
manipulation, circuit theory, nursing, and history, to cite 
only a few subjects). The student interface (student logic) is 
flexibly programmed and the student is free to get away from 
the stream of instruction into a free learning or enquiry mode 
in which he can access a variety of information sources. 

However, at a higher level of organization PLATO is 
uncommitted to any particular learning theory. If PLATO is 
regarded as a teaching and experimental facility, this fact 
allows for a heathy eclecticism (since it is easy to impose any 
structure). If PLATO is regarded as a complete CAI system, 
then the lack of specificity is a defect. 

(4) Smallwood’s System. The subject matter is structured 
as an ordering of modules which refer to conceptual 
structures and the programme is concerned with determining 
a path through this network. For this purpose it uses a 
learning model of a behavioural and stochastic type and a 
teaching model based on dynamic programming. The system 
is a feedback system insofar as behavioural indices are fed 
back to influence module selecting decisions and it is 
adaptive insofar as the decision structure is altered as a 
function of the students’ performance. 

(5) The SOCRATES System.* This system (or class of 

systems) is due to Stolurow and is designed on the basis of 
an “ideographic” technique. 13 The tutorial process is broken 
down into a pre-tutorial phase, a tutorial phase and an 
administrative phase. As in the other systems, the subject 
matter is pre-structured and the educational goal is specified 
at the outset. It is, however, clear that Stolurow intends to 
base his subject matter structuring upon a preliminary 
investigation of the learning strategies adopted by members 
of a population of students (essentially the preliminary study 
technique of 2.6(3)) and he has carried out experiments of 
this kind (notably in the field of syllogistic reasoning). The 
pre-tutorial phase of the CAI operation uses various tests, 
including specific and non-specific aptitude tests, to 
detennine the student’s initial competence. A teaching 
strategy is selected that depends upon their outcome and a 
structural (2.3(4)) learning theory. During the tutorial phase 
this strategy is executed and monitored by feedback 
information from the student and, if necessary, it is changed. 
The administrative phase involves the moment to moment 
housekeeping required in such a system. 

Thus, as 1 understand it, this system consists in a 
preliminary test (pre-tutorial phase) as a result of which some 
feedback, adaptive or conversational subroutine is tentatively 
selected. The operation of the subroutine is overlooked by a 
model that is not at the moment dogmatically specified; 
clearly, however, it is either a Cybernetic or a cognitive model 
and it might well be an artificial intelligence programme. 

* SOCRATES was set up at the University of Illinois. I am using 
this as a generic term for all of Stolurow’s work, including his later 
work at Harvard. 



Performance information is collected and may either be used 
to modify the set of available subroutines or to divert an 
individual student to a different subroutine. 

(6) The IMPACT system (Seidel and Kopstein) resemWes 
the SOCRATES system in its overall design. However, there 
are certain very important differences (in fact, IMPACT was 
independently devised and its relation to SOCRATES is 
largely due to the fact that both are big CAI systems, inspired 
by comparable objectives and ideas). 

(a) The operational core of the IMPACT system is an 
instructional decision model (IDM) which asserts what should 
be done when a student, his state of knowing, and the current 
state of instruction are given. In a limited context the IDM is 
the control rule in a conversational subroutine. However, it 
may be more broadly interpreted as the instructional 
sequence selector (the higher level control of SOCRATES). 
Thus, the IDM is the IMPACT Teaching Model. 

(b) The short-term educational goal of the system is to 
establish coherent and maximum information interchange 
between the student and the system itself (this goal is 
satisfied if and only if the discourse satisfied certain criteria; 
notice the marked departure from the behaviouristic 
paradigm and the obvious necessity for a rich interaction 
language).* 

(c) The long-term educational goal is to achieve a state in 
which (as indicated by properties of the discourse) there is 
symmetry between the state of knowing in the system and 
the student. 

(d) The IDM aims to satisfy both goals. It does so on the 
basis of a subject matter structuring and a student image. 

(e) The subject matter structuring resembles that used in 
the other systems (since IMPACT teaches COBOL 
programming, the “map’ 5 ' part is fairly tractable). It should be 
noted, however, that the IMFACT designers are the only 
people who have made a really serious attempt to deal with 
the subject matter structure, in particular to condense it, in 
terms of general systems theory. This part of the model is 
incomplete but some very promising progress has been made. 

(0 The student image is a direct, system representation of 
the student embedded in the IMPACT learning model. Part of 
it is a pre-tutorial (initial condition or component 3) 
representation and part of it is updated (component 4 of 
Section 2.2). Most of the properties used to specify the 
student image are fairly conventional (aptitude test data, 
latencies, mistakes, etc., taken with respect to specific 
exercises) but IMPACT is notable for its pioneering use of 
Shuford’s subjective probability or confidence estimates 
which will form an increasingly important data source ls the 
system is developed. 

(g) The rest of the IMPACT learning model is of the 
Bunderson and Guildford type (the statistical cognitive model 
of 2.3(12)). 

(h) The statistical data of (g) are related to the subject 
matter structuring and the input to the IDM (Teaching 
Model) by Structural «;.d Cybernetic Models (in the sense of 
Section 2.3) and by records of the students’ preferred 
learning strategies. Some of these strategies are built into the 
system as a result of preliminary experiments but some are 
learned (by the system) as the system operates. The 
communication between the student and the system is rich 



* These are evaluated. There is a clear sense in which IMPACT can 
reach a comprise with the student. 



enough to allow the student to access new strategies of his 
own invention and to add these to the strategy list).* 



2.8. Conclusion, Research Fields, Task Descriptions and 
Subject Matter Structures 

It is possible to specify competent tutorial subroutines for 
various tasks and further research in this area is needed and 
may be expected to provide an immediate payoff. If enough 
of it is done, we should soon have a framework in which the 
appropriate subroutine can be recommended for any aspect 
of education. 

Unfortunately, we can have far less confidence in the 
specification of a full CAI system. Several essays have been 
made in the direction of a general paradigm (as in 2.7) but all 
of them are marred by one outstanding deficiency. The 
subject matter structure and, to a lesser extent, the 
educational goal statement are arbitrary in the sense that 
there is no theory for saying what structure should be 
adopted. The matter is important because the subroutines in 
any of the CAI systems so far devised are held together and 
integrated by this structure. However, the problem can easily 
be overstated and it is worth giving some thought to the 
nature of the blemish and to just how “arbitrary” the subject 
matter structure really is. 

Miniature tasks involving a single tutorial subroutine also 
have a subject matter structure; At this level it is classed as a 
“task description” which the student is asked to accept. ** 
The description of the task and the educational goal is 
“small” enough for the student to assimilate at one 
intellectual gulp. Because that is so he might reject it instead 
of accepting it (i.e. “I didn’t see things that way” or “for all 
your blandishments, I won’t see things tliat way”). Because 
the student might reject the task description if it was 
altogether inappropriate, the task description is not tagged as 
objectionably arbitrary and, in general, it goes unquestioned. 

Although the subject matter structure of a full CAI system 
has exactly the same form as a task description, it is much 
larger. It cannot be assimilated all at once and consequently 
it cannot, in principle, be rejected by the student (at the 
most, the student can reject the task structuring up to the 
next subgoal ; essentially the task structuring imposed by the 
operation of a single tutorial subroutine). Hence, any 
structure that is imposed is open to the criticism that it was 
arbitrarily chosen (by convention, at the whim of a designer, 
etc.). 

In a way this is true. But I would like to emphasize that 
the subject matter structure (an ordering and specification of 
concepts to be attained) need be no more arbitrary than 
many similar constructs in the behavioural and social sciences 
that are commonly accepted without question. The fact is, 
the teaching system designer could set about discovering the 
natural subject matter structure in much the same way that a 
linguist sets about discovering a natural language or the social 



* Satisfied by rather sophisticated interface equipment and an 
ilab orate set of rules for comment, command, description etc. 

** We have already emphasis "id that the operation of a subroutine, a 
single control system, is game like. The game like or normative character 
of the tutorial interaction is, tor example, stressed in Section 2.6(2) and 
2.6(3) but is suppressed in the behaviourally oriented system of 
Section 2.6(1). 



anthropologist sets about discovering the kinship system of a 
strange tribe. In the case of a subject matter structure the 
task is eased because the culture is far from alien (the native 
informants are students and teachers) and the symbols and 
ikons have a clear meaning. On the other hand, the basic units 
are concepts rather than phonemes or words and the emphasis 
is upon pragmatic and semantic (rather than syntactic) 
relations between them. 

I do not propose this as the most economic or efficient 
method for determining a subject matter structure. But it is 
important to recognize that it is always a possible method. 
For, if the teaching system designer did adopt it, then his 
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The session began with Professor Gordon Pask in the guise 
of a ‘learning system analyst’ contributing a comprehensive 
overview of the theoretical bases of the learning process. 
Taking the point of view that teaching is a form of control 
and that teaching systems are built to control the learning 
process, he reviewed representative teaching-learning 
systems and discussed distinctions between the various 
types. Continuing on the thesis that any controller must 
be designed on the basis of a model for the controlled 
process, he took up several issues to do with learning and 
teaching models and related these notions to the business 
of computer based system design. His paper touched 
on much of the research going on in Great Brit/n and in 
the U.S.A., and in his introduction Professor Pask introduced 
illustrative material taken from his own recently completed 
experiments. 

Professor Pask was at pains to lay the myth that CAI was 
wedded in principle to the tedious paradigm of a question 
and answer routine; and that the educational goal of the 
control system must be fully and formally specified. Other 
points he made included discussion of control methods as 
educational facilities or as educational sub-routines; the 
problems of evaluating CAI systems as a whole; and the 
need to pay due attention to psychology and logic in 
designing computer based systems. In talking about 
learning models he stressed that these incorporated several 
separate representations, such as that of the subject matter 
to be instructed, the educational goal, the teaching system 
and its strategies, etc., and contrasted their descriptive 
(predictive) uses with their prescriptive uses. Finally he 
illustrated his analysis by case-histories of some fully- 
specified CAI systems. 

Professor Mitzel then raised four questions for general 
discussion by the group. (1) Psychologists have traditionally 
ignored individual difference variables (except mental 
ability) when studying human learning. How important 
will this set of variables turn out to be in building 
future CAI systems? (2) How important is it for future 
CAI systems to have a query capability for the student 
in order to approximate more closely live tutorial 
interaction? (3) In the evaluation of learning of different 
CAI systems, what is the impact of using norm-referenced 
achievement tests versus criterion-referenced achievement 
tests? (4) What degree of learner control of the general 
or typical CAI program is optimum in order to maximize 
both speed of acquisition and retention of information? 
These questions were followed by a brief discussion of 
conversational control systems. 

An intervention followed in which Professor J. Jones of 
Louvain University presented additional information on 
the automated teaching program which he had described 
the previous day He described the teacliing strategy and 
feedback mechanisms of the system which is designed 
for the teaching of physics to undergraduates at Professor 
Jones’s home institution. 

O Dr. Rothkopf asked about the possible application of a 




kind of Heisenberg Uncertainty Principle in attempting 
to diagnose a student’s particular learning situation. 
Professor Jones agreed that there was a variability intro- 
duced by the act of diagnosis. Professor Pask said that 
interrogation of the students does undoubtedly modify 
the student’s choice of a learning strategy. It does, he 
said, define the choice set of learning plans and, in his 
experience, learners invest in a plan and achieve a kind 
of “cognitive fixidity.” In the subsequent discussion, 
Professor Pask held that a conversational control 
system (in which the student participates in the learning 
strategy) is simpler and less expensive than a fully adaptive 
and automatic control system. Mr. Duke asked if there 
wasn’t a danger of overemphasizing the decision role of 
the machine. Shouldn’t, he queried, the teacher be 
furnished with information about the student and then 
the teacher make some of the adaptations in the 
instructional strategy? Mr. Hill raised a question about 
the adequacy of the information processing powers of 
the teacher and indicated that he doubted whether 
these powers were sufficient to allow a person to act 
in place of the machine. Professor Duncan Hansen 
related his experience at Florida State University in 
dealing with students on a CAI physics course. He 
pointed out that human beings are “least effort” 
machines and that they will typically find the path 
through the material which involves the least amount 
of time and of effort on their part. 

Dr. Eraut questioned what one was trying to optimize. 
There was danger of optimizing on the content and 
using the student strategy for this end; surely the main 
educational objective should be to help the student to 
choose the most appropriate strategy. Results of 
evaluations would also be markedly affected by the 
performance level selected as acceptable. A further point 
he made and on which several speakers commented 
was that since courses did not exist into which CAI 
could be fitted, how was a context to be formed to 
create information on which policy-makers could 
make decisions about implementation. Until better 
theoretical insights were available one could only work 
bv trial and error to establish a base-line and improve 
from there — however such was the bankruptcy of the 
evaluation procedures of classical psychometry that one 
did not know when a case became optimal and when to 
stop developing. 

The corollary was that management decisions had to be 
made on hunch today, but that neglect of this information 
would prove crippling to the development of computer 
based learning. 

The difficulty was whether the results from psychological 
experiments using non-meaning learning will apply to 
meaningful human learning which we hope to implement 
with the aid of CAI. 
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INTRODUCTION 

This paper will deal with the broad picture of resource 
allocation for research and development and implemen- 
tation of computer-administered (or computer-assisted) 
instruction, CAI. Specific cost projections w'thin an 
operational setting have been discussed previously 
(Kopstein and Seidel, 1968) and therefore will be 
touched upon only briefly. Resource allocation wi'l 
refer to funds, people, facilities, and the delegation of 
appropriate authority to formulate appropriate policy. 

And the fact that all these resources will be considered, 
viz., government, industry, and education, means that 
inevitably politics and economics will be intertwined in 
the subsequent discussion. 

The features of the presentation will be a brief 
description and justification for CAI as a technology. 

The need for incorporating a systems approach to 
educational innovation will follow to set the proper 
framework for the magnitude and complexity of the 
required research and development effort. An analogue 
to a corporate level of investment into research and 
development, 3% - 5 % of income, is proposed to effect 
the orderly transition of CAI from breadboard, through 
prototype to an operational system. 

Problems which arise in the course of considering 
appropriate resource allocation stem in part from the 
fact that none of the so-called entities, govtinment, 
industry, or education are monolithic. Government 
involvement will be both central and local. Industry in 
CAI includes hardware manufacturers, book companies, 
etc. Education includes central and regional 
administrators, teachers, research and development 
personnel as well as the ultimate user, the student. 
Problems of program management and co-ordination are v 
thereby made difficult. (The problem of educational 
value is also considered en passant.) 

Finally, a partnership model will be proposed to evolve 
operational CAI. The vehicle of a national R&D center 
with regional satellites is suggested to provide both 
proper trcLlng and to permit flexibility of research 
approaches to accomplish the goal of operational CAI. 

The context within which these points are discussed is, 
anything worth doing is worth doing right. 

Thus, we will ask if research and development into CAI is 
worth expenditure of money, time and effort. As you 
might predict , our answer will be affirmative. Granted 
this is so, what expenditures are necessary to do it right? 
Finally, how should the resources be properly allocated 
among industry, government and the educational 



community to bring CAI to fruition? Fruition in this case 
refers to the actual production of multiple copies of 
operational CAI systems usable in schools and universities 
throughout a nation. 

CAI is a Necessary Technology 

Various prominent individuals and agencies in the United 
States have indicated the need and desirability of exploiting 
the educational opportunities inherent in individualised 
instruction. The Committee on Economic Development has 
represented the broadest cross-section of U.S. society in 
its appraisal (businessmen, educators, psychologists and 
community leaders). In essence, they present the case well 
for a failure of our current educational system to take 
advantage of modern technology and to deal effectively 
with the increasing requirements of our complex society. 

The Committee has pointed out that ‘individualized 
instruction geared to the individual interests, abilities and 
learning rate is one of the cherished goals of American 
education. It is an aspiration which we wholeheartedly 
share, yet the schools are making very slow headway in this 
direction through present means ...’ Psychologists have been 
engaged in the study of individual differences for years. 
Teachers have long been complaining of the inability — 
administratively — of coping with students as individuals and 
have been thereby forced to teach, for the most part, to the 
mean of a class. 

Our view is that CAI is the leading, operationally defined 
edge of a model of individualized instruction. It represents 
the potential (with all due respect to Oettinger and Marks’ 
objection) for a quantum leap in adapting instruction to the 
momentary needs and capabilities of the individual student. 

It provides the basis for an iteratively improving instructional 
environment. A word should be added here about the 
distinction between technique and technology. The criteria 
for evaluation of each differ. A technique may be properly 
compared to another technique with the same system model. 
Teaching reading by the phonics method versus ‘look and 
say’ is an example of such a comparison. Indeed studies have 
been conducted attempting to hold all other components of 
the momentary education system constant save the difference 
in technique. 

In evaluating a new technology, however, such relatively clear 
and simplistic comparisons are insufficient. It may well 
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happen as in the case with CAI and individualized instruction 
in general, that the system in question must be redefined. It 
is silly to mold a computerized, individualized teaching 
environment around a 6 hour day. It is also improper to 
consider the economics of the new technology in terms which 
neglect the new opportunities opened up and hitherto 
impossible. To quote Harley (p.56, 1967) ‘When economies 
result [from using the new technology] they are derived from 
the reduced cost of providing additional services — from the 
improvement in the quality of teaching and the level of learning; 
from the ability to shrink time and space, and from the sharing 
of limited resources. 

CAI is thereby a technology, not a new experimental classroom 
technique. Indeed, the magnitude of societal effect possible 
with CAI without any hesitation may be made comparable to 
that of the industrial revolution. The potential for basic 
improvements in educational systems exists but it must be 
brought to the proper climax. The economic exploitation, 
frustration, disruption of employment, suicide, etc., which the 
industrial revolution had led to, could have been avoided with 
proper planning. With the advent of the computer, automation 
has changed man’s role already to that of innovator rather than 
routine performer. On a purely rational basis, therefore, CAI 
is worth doing wherever individual differences make a 
difference in the field of instruction 

CAI Research and Development is Complex 
Having answered — all too briefly — our first question affir- 
matively, i.e. CAI is indeed worth doing; the second and 
more thorny question must be addressed: What do we mean 
when we say, ‘It should be done right’? We will first 
present in Figure No. 1 the Eight Steps which comprise a 
total systems approach. 

The Eight Steps of the Systems Cycle 

1 . State the real NEED you are trying to satisfy. 

2. Define the educational OBJECTIVES which will 
contribute to satisfying the real need. 

3. Define those real world limiting CONSTRAINTS which 
any proposed system must satisfy. 

4. Generate many different ALTERNATIVE systems. 

5. SELECT the best alternative(s) by careful analysis. 

6. IMPLEMENT the selected altemative(s) for testing. 

7. Perform a thorough EVALUATION of the experimental 
system. 

8. Based on experimental and real world results. FEEDBACK 
the required MODIFICATIONS and continue this cycle 
until the objectives have been attained. 

Figure 1 



This framework (with slightly different words if you like, 
or 7 versus 8 steps) must be followed from research through 
development and implementation in order to result in 
operationally valid and useful CAI systems. We should add 
that this approach is also necessary when attempting 
improvements in a traditional instructional model. We 
shall return and enlarge upon this approach later when 



proposing a model for accomplishing proper CAI develop- 
ment. Its importance is amplified tenfold with an 
innovation as encompassing as CAI can be. Note then the 
framework does not preclude CAI as drill and practice, 
simulation, or use of a strictly calculational aid for problem 
solving. These can all be useful parts of an overall CAI 
R&D project. However, as I have previously asserted, the 
instructional system consists of N components. If only 1 
or 2 components are clearly identified, measured and 
controlled, their contribution to instructional output 
relative to that of all the N-l or 2 other uncontrolled 
components becomes extremely difficult if not impossible 
to assess (Seidel and Kopstein, 1968). 

‘Doing it right’ thus means having a careful system research 
and development effort integrating the facets of computer 
hardware and software with content development and 
studies of alternative instructional strategies. This means 
in turn a multi-disciplinary effort. If CAI is to fulfil the 
promise of delineating the relative importance of the various 
characteristics pertinent to appropriate decision making in 
instruction, it must have the opportunity to model the 
instructional situation, to vary parameters and to apply 
these to meaningful human learning. 



The Magnitude of Required Effort for CAI R&D is Large 

Let us ask the question at this point what experience has 
shown us to date have been the requirements of any large- 
scale CAI centers for annual budget requirements. From 
informal discussions with knowledgeable persons regarding 
their CAI projects’ fiscal problems, it appears both inter- 
nationally (Mr. Kirchberger of France) as well as in the 
United States that an operating buoget of approximately 
$250,000 to $300,000 a year is necessary merely to maintain 
facilities in operations. The reason for this becomes quite 
clear if one considers simply the rental price of an IBM 1500 
system as an example. The hardware alone averages $100- 
1 10K per year. Taking the $250K total, thil leaves $15v,K 
per year for staffing, administration, and support facilities. 
The result is that, given approximately $38K cost per 
professional man year, one can have only a minimal CAI 
program (4 professionals) concentrating on operational 
activi^es. 

How much then should a total systems approach toward the 
development of CAI require to produce an operational, 
useful example of individualised instructional models? We 
cannot answer this in absolute ter ns, but on the basis of 
our experience with Project IMPACT’S multi-disciplinary 
staff involving 18 professionals and 7 others in supporting 
roles, a personnel budget of at least twice these amounts 
seems necessary, Gur installation is unique in that it is 
funded, so far, by a single arm of the government. Of 
course, one of the problems for most of the CAI projects 
is that they are funded from multiple sources with 
different der ands; and in order to satisfy the requirements 
of the various funding agencies, the research and develop- 
ment is fragmented in many different directions. Until 
recently the awareness of funding at a ‘critical mass’ level 
did not seem to exist. Recently it seems that- the Office 
of Education in the United States has taken steps to support 



fewer projects in CAI but at a higher level of funding. At 
least tliis seems to be a move in the right direction. The 
problem, however, to maintain this critical level is to 
permit not simply existence of an operating environment, 
but rather to facilitate large-scale, integrated centers to 
study and arrive at both the adequate descriptions of 
learning processes and the necessary prescriptions for 
instructional development. 

The most recently published figure from the Bureau of 
Research in the U.S. Office of Education ( Educational 
Technology, April, 1969) indicates that they are providing 
an average of roughly $287,000 per project over ten CAJ 
projects. If other governmental agencies could support 
these same projects, w'thout changing their goals, with a 
comparable amount of money, then it would seem to be 
possible to proceed beyond the breadboard stage of CAI 
development. One point of clarification is worth noting 
at this moment with respect to encouraging flexibility 
and diversity in approaches. This promotion of varying 
approaches to solving the strategy development and 
overall construction of useful CAI should not be in 
any way misunderstood as promoting the support of 
multiple small-scale efforts. Furthermore, if a project is 
required, once funded, continually to submit and resubmit 
a multiplicity of proposals, a large amount of time and 
effort will go into non-productive work. What is required 
is that a rational basis for selection of project proposals 
should be used at the outset, and then a reasonable amount 
of funding be provided over at least a five-year period. 

Without advocating that all the eggs be put in one basket, 
we are saying that the highest rate of progress is likely to 
result from a distribution of available funds to fewer 
baskets at sufficient levels to permit large-scale integrated 
efforts. On the other hand, funding a diversity of projects 
at very small levels of funding will most likely result in 
none of the efforts ever getting to the point of providing a 
full-fledged operationally, implementable CAI system for 
education. 

An Analogue to the Corporate 3 %-5% Investment 
is Advocated 

Next, what expenditure should be invested in a CAI total 
systems effort? We can start by taking a cue from a 
corporate model and note that a number of sources 
(recently Duckworth of the United Kingdom, 1967, as well 
as representatives of corporations in the United States) 
have indicated a 3% — 5% level of corporate income is 
appropriate for these purposes. In a recent talk at the 
American Educational Research Association, the branch head 
of special projects in the Office of Computing Activities of our 
National Science Foundation estimated that last year the 
entire educational enterprise in the United States cost some- 
where around $50 billion (Melmed, 1969). He gives a figure 
for expenditures on educational research which approximates 
$100 million, or in other words a .2% investment. Even with 
a large error factor, it nevertheless appears ‘that the investment 

in educational research is only a fraction of 1% of the 
educational enterprise.’ In contrast to this, R&D 
investment in the electrical conununications industry last 
year stood at approximately 3.4%. Secondly, he alleges 
that IBM invested $300 million, or roughly 5%, in R&D of 
its reported $6 billion gross income for the last year. 



If we accept the 3% - 5% figure as necessary to sustain 
viable R&D in education, the annual dollar investment 
given the $50 billion total year expenditure should be 
between $ 1.5 and $2.5 billion. Before you wilt under the 
apparent enormity of this amount, consider it relative to 
innovative programs with which we are more familiar. The 
United States has just put a man on the mocn. Our under- 
standing is that wo have spent $24 billion and 10 years of 
research and development to go from the breadboard stage 
through the prototype to the ultimate system for the 
lunar landing and retrieval. Accepting the 3% -5% figure, 
a comparable time period for innovative developments in 
education would amount to an investment between $15 and 
$25 billion dollars. The question we ask you to consider is, 
is education any less valuable than a lunar landing or space 
travel in general? Our personal reaction is that both efforts 
are extremely valuable. Forgive us if we seem to go off on 
events within the United States, but these are figures and 
programs with which we are much more familiar. You can 
readily apply the percentage estimates to your own. country 
considering your own expenditures, gross national product, 
etc. 

In any event, let us carry the analogy one step further. The 
dollar cost of investment in education, in particular in CAI, 
further pales by comparison to the costs of other single 
technological developments. For example, the United 
States government’s contribution so far to the controversial 
supersonic transport program is roughly $1 billion. The 
cost of the deep-dive nuclear research sub is roughly $99 
million. Many more such examples could be cited. Note 
that these are specific system efforts. We have previously 
documented that an implementable computer-administered 
instructional system, following from breadboard through 
operational development stages, could be accomplished for 
something in the order of $5 million in a careful time- 
phased schedule (Seidel, 1969). We would submit that this 
is certainly not off by more than a factor of 2, and further 
that the value of such a development, as we have tried to 
indicate, can be monumental in reshaping the whole field of 
education. You recognize, of course, that this must include 
adequate interdisciplinary staffing and evaluation and testing 
of all the components of experimental hardware, computer 
software, instructional strategy and various selected subject 
matter. However, we submit again that the expenditure is 
but a minute fraction of the cost tor other technological 
developments. Surely the goals are at least equally 
desirable. Let us submit further that given the 3% —5% 
investment in all of education, enough money should be 
available for such innovative efforts as CAI to permit, as 
Oettinger has proposed (1969), a great deal of flexibility and 
diversity in approacnes. It thus should be possible to solve 
the educational technological problems of using the computer 
properly in the entire field of instructional and educational 
improvement. 

Problem of Interpreting the Value of Education 
To return and answer some of the rejoinders which have surely 
been developing as you have been listening to what we have 
been proposing as a reasonable level of dollar investment, let us 
consider the fact that the goal of a space program such as 
putting a man on the moon and retrieving him is quite specific. 
The parameters for guidance, for control of the entire system 



required to perform this feat, although large, are finite, 
measurable, quantifiable. A major problem for education is 
to identify all those factors which are pertinent to estimating 
requirements for specified achievement characteristics, and 
also for interpreting the value of that educational achievement. 
Another paramount problem that educational technology 
faces is opposed to a neat and closed engineering technology 
such as the example of the space program is that the return 
on investment on education may not be visible for roughly an 
entire generation. The attainment of objectives in the space 
program, on the otner hand, was relatively immediate, 
dramatic, quite tangible, and rapid. Our only rejoinder to 
that is that in educational technology, one must extrapolate 
from immediate criterial effects of measured educational 
proficiency to ultimate criteria of general societal signifi- 
cance. Of necessity this means at the outset that we have 
clearly identified the parameters, our educational inputs and 
our educational outputs. 

We must not permit our experience with the ambiguities and 
vagaries of the traditional educational instructional system 
to force a premature and inappropriate evaluation of 
computerized instruction. True, there have been difficulties 
in measuring a ‘good’ teacher, certainly in traditional 
instruction one of the most significant educational inputs 
(Froomkin, 1969). But, as noted previously (Kopstein and 
Seidel, 1968; Seidel, 1969a; Seidel, 1969b) in computer- 
administered instruction, we can objectively document the 
dimensions of that system. Cost effectiveness can even- 
tually be measured. The instructional agent’s value (instead 
of the human teacher’s) can be measured against its output 
(student achievement). The costs of the entire CAI system, 
input, transformation and output, can be evaluated and 
justified or not, upon tangible and objective bases. 

What then are the implications for consideration of cost 
effectiveness studies in CAI? Granted that we may extra- 
polate to later generation returns, it certainly seems >ve 
must consider direct outputs of the instructional system if 
our evaluation is to have any substance at all. Standardized 
achievement units must be the criteria (see Randall and 
Blaschke 1968). Harley (1967) in an insightful appraisal 
makes the point even stronger. *... we have been concerned 
with the cost per student taught (our input) and not the 
cost per student learned (our output) yet. we know that what 
is taught and what is learned is not synonymous.’ We have 
ignored therefore the vital ‘... cost of our scrap (non- 
opeiable units) — students who cannot function in today’s 
society ...’ and therefore our cost statements have fallen 
far short of the true costs oi our education system (page 52). 

A host of other problems beyond the scope of the present 
paper relate to establishing other measurable values of 
education. For instance, Suppes (1968) has recently called 
for ‘clearly stated normative principles’ (page 12) for dealing 
with the contradictions in modern pliilosophy of education. 
While not a direct concern of economics and CAI, certainly 
resolving the existing antonomies of education d philosophy 
(e.g. maximum freedom of choice etc, versus development 
of a sense of discipline, content versus method, social 
adjustment versus maximum achievement) is essential for 
a meaningful evaluation of any educational innovation. 



This is particularly critical (perhaps propaedeutic) for CAI 
which depends upon acceptance to a large degree of the 
model of individualized instruction. We do not wish to 
dwell on this except to point out that it is an extremely 
important problem to be resolved. 



Problems Exist in Establishing a Potential Partnership Mode! 
The next item is equally important to establishing that 
CAI is valuable, and to establishing the necessary dollar 
figures for R&D investment. That is, what kind of a 
workable framework can exist, within which this research, 
development and operational environment is appropriate? 
Can industry, government and the educational profession 
be combined to bring the effort to fruition? What policies 
must be incorporated thereby to bring this about? How 
can resources, money, facilities, personnel and time be 
most appropriately allocated? 



Are Goals Compatible? 

Before delineating the proposed prototype for the three- 
way partnership, let us examine potential problems that 
may arise from the combination of industry, government 
and educational professions in this manner. One can ask 
if the model of research, development and operational 
utility for industry is compatible either with the educational 
profession or that of the government. Interestingly enough, 
we already have at least one failure in an attempt to bring 
these three entities together. In the United States there 
had been a Project ARISTOTLE intended to be a 
catalyst and continuing stimulant for educational innovation 
through these three arms, and it has been dropped for lack 
of support. Perhaps the answer lies in incompatible models. 
Industry in the United States exists to manufacture and sell 
at a profit. Its investment of 3% - 5% in research rests 
cleariy in the belief that the return will occur in a 
relatively short period, perhaps two to three years. This 
premise becomes muddied when we attempt to apply this 
industrial template education. Recall the generation lag 
for evaluation purposes in educational products that I spoke 
of earlier. 

Considering the educational model, the goals of education 
have never been defined in terms of profit and loss 
statements. Selling a product has never been part of the 
system. In fact a frequent criticism in recent years has been 
that the intellectual aspects of education were being sub- 
verted for more specific and practical occupational training. 

It almost seems that the traditional roles of industry and 
the educational profession are so antagonistic or at least not 
overlapping that throwing them together without creating 
a new compatible model for R&D purposes in CAI is to 
create an anomaly doomed to failure at conception. 
Moreover, government and education are interested ir. 
welfare of the populace, education being - in this sense — 
a subset of the governmental function. Industry is concerned 
with product development and sales. But need these be 
different? (Galbraith has previously said in The Affluent 
Society that the private sector could not handle this type of 
effort on its own, but why not a partnership?) 
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‘Government, Industry and Education’ are not Monolithic 
Entities 

For another thing, the so-called education ‘industry’ in CAI 
is not monolithic and not coordinated. It involves hardware 
manufacture, book companies and computer software 

houses at the very least. The companies are relatively 
independent with their own profit and loss statements, 
corporate policies, etc., and they all must be brought together 
in order to construct a meaningful CAI system. The edu- 
cational community and the government are also made up of 
components with differing capabilities. ‘Educational 
community’ consists of at least administrators, middle 
management in the person of assistant headmasters (or 
assistant principals) and the teachers on the one hand. On the 
other hand is the ultimate user, the student. Government is 
represented by central and local spheres. For funding we 
require federal and for active participation in implementing 
CAI, we require local involvement. 

There inay well be many objections to a proposed cooperative 
set of overlapping functional relationships among industry, 
government and the educational community. Not the least 
of these is an abhorrence at attempting to apply the 
corporate model of R&D to education, but we raise the 
question, why not? It has produced inventiveness, increased 
profits, and viable, new products in the market place. Why 
not ask the same of education R&D? 

Will the corporate model apply /If education is to be 
considered like industry, it is peculiar in that the financial 
resources are clearly not within the system perse but must 
be provided by outside sources - the local, regional and 
central governments. Recently {Educational Technology , 

May 1969) this has been called into question by those who 
would not like to apply the corporate model; but the 
alternatives are not being readily accepted (they imply clear, 
behavioral definitions of educational objectives, etc.) by 
the educational community. 

This difficulty is epitomised in a statement by a noted special 
assistant on education in the former U.S. presidential 
administration - ‘Perhaps the most traditional . . . , and the 
one (local community) most resistant to outside change has 
been the educational community.’ (Kearns, 1969). The 
purpose of the Elementary and Secondary Education Act of 
1965, Title III program, is to provide federal grants directly 
to local schools *.#. for the very purpose of stimulating 
innovation and change in local education patterns.’ The 
prevailing mood, however, has been deap-seated suspicion in 
the United States that federal aid means federal control.- 
Consequently, a very serious-question is to be answered. Even 
given sufficient funding resources, how can a workable 
cooperative model be developed to ensure (a) valid R&D 
involving the educational community, and (b) proper imple- 
mentation in local systems? Part of the solution must of 
necessity involve commitment of dollar and personnel 
resources toward large-scale R&D efforts in CAI and toward 
massive training and retraining in the educational community. 

Let us turn the question arefund and ask it again. Is education 
becoming an industry? Should- it be? The problem would seem 
to be one of retaining the goals of intellectual expansion, 
freedom and innovation while adopting where feasible a model 



of improvement in education drawing upon the techniques 
of industrial development. What also seems to be required at 
the very least are policy guidelines from the central government 
to force a necessary workable structure. This would, 
providing the policy is appropriate to the task, aid not only 
industry, but the educational profession and the educational 
user, the student. 

A Proposed Partnership: Allocate the Unique Resources 
amongst the Steps of a Systems Approach 
It should be clear from the foregoing discussion that we are 
faced with an enormity of (a) financial investment, 

(b) the requirement that this effort serve the public welfare, 
vis-a-vis the educational consumer, as opposed to a private 
corporate entity and its stockholders, and (c) that the return 
on investment may in fact take much longer in accurately 
calculating than what industry’s profit and loss statements 
demand. All of this indicates that the central government 
then is the only one that can adequately and properly provide 
the funding the policy guidelines for this type of partnership. 
Secondly, the techniques of industrial research and develop- 
ment (that is, product development) are appropriate to 
contribute to an evolving CAI system. Various industries 
have the facilities and the tools for development and 
production (certainly for the necessary hardware and soft- 
ware components in CAI). The educational community is a 
unique partner in the sense that the local school system and 
its personnel are going to be the focal point of the develop- 
ment but they do not have the financial resources nor do 
they have more than a portion of the total personnel required 
to accomplish this innovation properly. Their partnership 
contribution consists in developing instructional content 
through providing subject-matter experts as part of an inter- 
disciplinary team, and by providing the demonstration and 
test facility within which the innovative experimentation 
and development can take place. 

Finally, in addition to the partnership structure, the 
developing CAI effort must include, as I have indicated 
earlier, a functional approach which embodies systems 
design and total systems development using the 8 steps. Now, 
how does this apply to CAI? The discussion of the steps is 
worth repeating in slightly different form if we are going to 
demonstrate a workable model for this partnership. 

To create a proper operational CAI system, it must pass 
from the breadboard stage, through the prototype, to an 
operational, cost/effective phase. ‘Breadboard’ is a term 
originated by electronic circuit designers. During very 
early stages of developing an electric circuit the paper 
design (conception) is translated into a set of components 
provisionally connected (by alligator clips and a few wires) 
and tacked onto a wooden board. The purpose is to verify 
that the design scheme will have the general characteristics 
expected of it. By extension ‘breadboard’ refers to any 
first and provisional realization of a system design. 

Applied to CAI, documents like the CEP report have stated 
the NEED (see pp. 3 & 4) in education for exploiting 
individualized and personalized instruction. We have defined 
our OBJECTIVES in terms of producing student output 
at a given level of achievement. As part .of the design note 
that this process is to be an iterative one in attaining that 
goal. In the breadboard iteration, only the most crucial 



design criteria are applied. Secondary objectives, i.e., 
desirable features or ‘nice-to-have’ characteristics, are kept 
from confusing the basic design problem. For example, 
with reference to CAI, at the breadbroad stage it is 
inappropriate to consider timesharing the computer with 
batch-processing operations. Similarly, during the breadboard 
stage of design, operating CONSTRAINTS are minimized. 

In terms of CAI, it may be essential to develop inexpensive 

student terminals, but first terminals with adequate 
characteristics must be designed. Further limitations stem 
from available computers and compatible CAI equipment 
and languages. Current CAI systems are divided among those 
which have not progressed past the breadboard stage and 
those which have tried (unsuccessfully) to by-pass this 
stage. ALTERNATIVE instructional decision-making 
strategies and mixes of hardware and software subsystems 
with selected subject matter must be considered. Following 
systematic evaluation and study of the alternatives, 
SELECTION of an initial system is made and IMPLEMENTED 
for a test run. EVALUATION is made based upon student 
output. FEEDBACK to improve (modify) iteratively the 
CAI system, viz., meeting objectives, is made. 

This process continues throughout the iterative development 
in order to refine the system. Once the breadboard phase 
has been completed it is possible to proceed to a prototype 
system design, the circumstances under which the system 
must perfoim. In this phase one first establishes precisely 
what the system is to do and major constraints such as 
permissible costs or delivery time are also taken into 
consideration. Various available means (e.g. magnetic or 
optical information storage) are weighed against 
optimization criteria. Optimization means a best compromise 
among contradictory objectives and imposed constraints. 

(e.g. lightweight, portable student terminals with character- 
video-audio display capability for no more than $500 per 
unit) in terms of some ordered set of criteria (price more 
important than display capability which is more important 
than portability). Finally, the design plan that has emerged 
is implemented and a first prototype is synthesized. A 
prototype CAI system may have operational usefulness, but 
is likely to include design flaws and oversights that ought 
not to be multiplied in many duplicated installations. A 
prototype is merely an untried and unadjusted assemblage. 

A tested system emerges over .a number of subsequent 
repetitive development cycles. In the case of a CAI system 
only a small number of students would be exposed to its 
instruction initially. Their interactions with the system must 
be minutely monitored and appropriate adjustments made. 
Massive data need to be collected from which it can be 
determined whether the prototype is actually performing 
as envisioned. Where actual and expected performance 
disagree (e.g. , mean delay of system response to student 
exceeds stipulated value of 1 second) revisions must be made 
in the system design so as to bring them into line. A tested 
system exists only after actually observed system performance 
coincides with expecte d performance . Only then is it 
economically justifiable to use the prototype design as a 
template for multiple reproductions. 

To place this approach within the partnership framework, 
the initiation of policy and guidelines (statement of NEED) 



would come from a central government agency. (It is 
conceivable that in a somewhat different form specific 
needs might arise from statements within the educational 
community.) 

These would be transformed into specific system OBJECTIVES 
jointly by team members representing all these arms of the 
partnership. An interdisciplinary research and development 
effort (comprising the remaining steps except EVALUATION 
and FEEDBACK) would be conducted with the lead role 
most probably taken by a non-profit (rather than a profit- 
oriented) R&D corporation (see next section for reasoning). 

Local government and educational system personnel and 
facilities would provide administrative aid. Industry would 
fabricate the necessary hardware and software, and members 
of the three-way team would then attempt to IMPLEMENT 
the provisional system. At this stage, the work primarily 
would be accomplished by industry and educational 
community with administrative aid provided by local 
government. 

Considering the possibility of this joint venture as 
indicated, one more link must be added. To provide the 
necessary objective EVALUATION and FEEDBACK in a 
coordinated manner, an independent fourth party is required. 
The form of the feedback information would be appropriate 
to the particular partner of the three-way team in order to 
make appropriate modifications to those aspects of the 
systems development process uniquely under its jurisdiction. 
For example, if modifications were required regarding 
statements of need, this could be provided to the policy and 
guideline process for the government. As the modifications 
were necessary in fabrication of equipment and software, it 
would be given in unique form to industry, etc. While this 
may be difficult to conceive of within a given country, it may 
not be far off from what is currently proposed by the OECD 
group (Organization for Economic Cooperation and Develop- 
ment), an international body to establish policy and make 
apolitical, dispassionate and objective evaluations of CAJ 
studies, in the U.S. fragmented examples are beginning to 
crop up. 

A Vehicle for CAI Research, Development 
and Implementation 

The last item we wish to suggest is a definite means by which 
the proposed partnership could proceed efficiently. Within 
a country and in particular the U.S., there are two such 
vehicles. We propose first the establishment of a national 
center for research and development on innovation in 
educational systems. 

This is a necessary institution to establish, in fact, 
and has been proposed previously for the"Navy Department 
in our country but was not funded by the Department of 
Defense. The Center’s prime function would be to 
coordinate the application of diverse scientific and 
technological principles in the solution of educational 
problems and, generally, to evolve educational technology to 
higher and higher levels. At any rate, this centre would 
have closely tied regional satellites which would carry out 
the translation of the results of the research and develop- 
ment into operational reality within local school systems. 



The fourth party, "VALUATIVE entity, which I described 
earlier fits here. It would be this entity which would bring 
together the industry, central government funds, and local 
governmental and educational systems for reorienting 
personnel into a demonstration program. 

At the national center a training emphasis would be given 
to pre- and post-doctoral levels in order to develop increased 
national competence in instruction theory. The areas to be 
included might be illustrated by Computer Sciences, 

Behavioral Sciences, Applied Mathematics, in general what 
we might call educational-technological research, technical 
writing programs, etc. On the regional level, the regional 
centers would train for tasks relating to the implementation of 
CAI, that is orienting communities, training local officials, 
local administrators of programs, etc. It is important to make 
clear that the national R&D Center will not be the sole 
technical, or active research, installation. Research and 
development would be carried on at local installations to 
permit flexibility in approaches and there would be technical 
feedback a input in both directions to upgrade the R&D 
status in general. A plan for coordinated activities amongst 
the regional and the national centers would be essential, 
else fragmentation and lost efforts (such as seems to be the 
case currently with the USOE-sponsored regional laboratories) 
would result. 

To accomplish testing and implementation there already 
exists a potential prototype of a fourth party in the U.S. It 
is a non-profit corporation (the Institute for Politics and 
Planning) which has the role of bringing together government, 
industry, and educational systems. This takes the form, for 
example, of establishing advanced learning centers in 
conjunction with the government, in this case not only the 
centra] government as the funding source but the local 
governmental groups as well as the local educational system 
participating as members of a testing and evaluation team. 

(We might add that the way this program is established 
currently the protit-mmded industry operates on a fixed-fee 
basis and gets paid if and only if students succeed in reaching 
established achievement criteria.) 

This effort is somewhat premature with respect to CAI 
because it assumes that we can already accomplish CAI 
implementation on a large scale. Nevertheless, a non-profit 
corporation may well be the focal point for making a 
partnership viable regarding all the activity phases, 
research, development and implementation. In the non- 
profit corporate model goals tend to be more directly 
oriented toward the welfare of society than those of the 
profit-maker who must put survival first. Alternatively, it 
may be possible to adjust the goals of the profit 
corporation. If the return on investment cycle were 
extended (to five or more years) through the provision of 
some type of government ‘insurance’ , profit-making 
industry could play a role equivalent to the non-profit 
entity. The ‘insurance’ debt could be retired at the end of 
some agreed period either out of the corporation’s 
assets (if no marketable product was delivered) or as a 
percentage of the profits derived from an implemented 
product. This at least represents one mechanism to consider. 

In any event, the partnership model would have to 
encompass all the facets of research, development and 



implementation. Too frequently, an innovation is 
developed and implementation is given short-shrift in the form 
of simply a written recommendation. To quote Mr. S. Clark 
Beise (Chairman, Executive Committee, Bank of America 
N.T. & S.A.) in the report on innovation in education from 
the Committee for Economic Development. ‘One of the 
major problems inhibiting change in our present educational 
programs and processes is the lack of communication between 
educators, teachers, administrators, school boards and the 
public.’ He goes on to note that the statement in the report 
by the CED develops a program that should be accomplished 
for research in innovation but does not carry the 
recommendation through sufficiently ‘to the point of 
being able to demonstrate their value to those who must be 
convinced that changes should be made. In order to 
disseminate information on recommended changes effectively, 
there should be established a system of demonstration 
schools, reasonably available geographically, to show what 
can be done in general practice to implement and integrate 
the recommended improvements within practical costs, into 
a rounded program.’ (Italics ours). We take what Mr. Beise 
says as to be reflected in the needs of CAI efforts 
internationally. 

We trust also that this discussion has demonstrated that ‘CAI 
is worth doing’, and that the approach proposed herein has 
suggested a reasonable framework for ‘doing it right’, We are 
certain that alternative proposals can and will be made to deal 
with the complex problems of resource allocation for CAI 
research, development and implementation. The rought 
sketch given in this paper can be viewed as no more than an 
opening wedge. However, we are equally certain that some such 
approach must be put forward and developed as a workable 
model if CAI is to live and fulfil its vast potential; otherwise 
this promising technology will die on the vine. 
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DISCUSSION 



Dr. Seidel opened the session by amplifying the material in 
his paper and giving some indication of the nature, 
objectives and costs of his own project. At HumRRO he 
has an operating budget of some $250,000 and a team of 
some 18 specialists. He noted that a CAI system has to 
develop through the stages of breadboard, and prototype 
before becoming a production system and estimated that to 
produce a prototype would require some $5M over 5 years. 

It is essential that this funding is fully committed in advance 
both as a hedge against inflation and as a guarantee that 
funds will be available. 

At present, he felt that industry, in the sense of profit- 
making industry could not take the leading role in CAI 
developments because open-ended systems were required. 

Dr. Seidel illustrated multi-point his model of CAI R & D 
given in Fig. 1 of his paper 

He emphasised very strongly the iterative, cyclic nature 
of this R & D model and the necessity for strong and* 
definite feedbacks. 

Dr. Adams critised some of the points made in the 
position paper. “Bob says that CAI research costs an awful 
lot of money so let’s go out and spend it. $100M is quoted- 
lot of money -should be able to slaughter problems.” He 
pointed out a contradiction in that it is implicit in the 
reasoning behind educational innovation that the present 
establishment has done a bad job, yet it is the present 
establishment who have to provide the research funds. 

He doubted that any major projects could be fully funded 
in advance or have a blank cheque. He suggested, taking a 
cynical view, that it was unwise to try and pioneer in any 
project, since the more the project claimed to be doing, the 
more political management it required and the more people 
had to be squared. It was better to choose an application 
which you were certain would survive, and concentrate on 
systems which were cmdely useful from the beginning. 

Dr. Rothkopf noted that there were two technologies 
involved in a CAI research enterprise: computers and 
pedagogy. When allocating resources to computers it is 
essential also to allocate them to pedagogy, if only to 
protect ourselves against present pedagogy. Clear ideas have 
not come out of current projects in the way everyone seems 
to assume. To balance all the $1M projects there should be 
a S50M project systematically evaluating the $1M projects. 
This could well be modelled on the Rothamstead Agricultural 
Station and its systematic exploration of agriculture. We need 
a long-term program of research into education. However, 
no-one will listen-they want their blue-painted boxes in 
the comer! 

Mr. Hill stated that institutions do not solve problems- 
what solves problems is taking action. People spend money 
on what they want and not what they should have. They 
require a pay-off and cannot see the value of long-term 
research. 

Mr. Broderick stated, “I am one of the few people here 
who work for a Local Education Authority, I am, therefore, 
probably more especially aware of the problems that will be 
involved in implementing computer-aided learning in the field 
in the United Kingdom. The biggest problem, as always, will 
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be money and I shall assume the unaccustomed and 
uncomfortable mantle of a Chief Education Officer.” 

It is necessary for a Chief Education Officer to make out a 
case for the use of CAI to ^ lay Council. This case must be a 
good one, if he requires £50,000 extra it will be necessary for 
him to ask the Council to sanction an extra Id rate and rates 
are highly-charged political issues. The Chief Education Officer, 
however, must make out his case, not on the grounds of 
politics and prestige, but on the grounds of social and 
educational benefits. What are the possible grounds he 
could use? 

(a) “Individualized instruction”: this is true, but if 
individual student terminals are to be supplied, this 
could be a very expensive exercise. 

(b) Computer-managed instruction: he stands a better 
chance justifying this if he can use computer equipment 
currently owned by tire Local Education Authority. 

(c) “Enrichment of education and extending the 
experiences of the Student”: here I am really referring 
to use of CAI in a gaming mode. This can appear to be 
one of the most valuable applications educationally, 
but the idea of enriching education may well be a 
difficult one to sell. 

(d) “Deprived areas”: chronic staff shortages, be the causes 
either sociologically or otherwise, would probably 
make a strong case with the Committee, there are 
already many schools that have serious difficulties in 
obtaining Mathematics and Physics teachers. 

(e) Fewer teachers will be required: this is blatantly untrue 
and at last Education Committees are looking to their 
Chief Education Officers to justify such claims by 
actually reducing the number of staff employed. 

(f) For use in remedial work: quite a strong case can 
probably be made out for this application, for remedial 
teaching is an area where it is difficult to get 
educational staff and which can cause minor chaos 

in school organization. 

(g) “Raising the base level of education”: a strong case 
can be made here if the current base level could be 
defined. 

In view of the points above, I recommend that research 
workers in the field considering the school area of application 
for CAI should bear in mind the climate in which their 
products are likely to be implemented. 

MissP. M. Ash noted that, “Comments have been made 
that the educational system has failed.” What measures have 
been applied which lead to such a conclusion? How, in fact, 
can success or failure in education be measured? Certain 
developments here suggest success: 

1. Selection for different types of secondary schools at 
age of 1 1+yrs. has been called in question largely due 
to the efforts of teachers in Secondary Modem schools 
who have shown that they, as well as the Grammar 
schools, have pupils capable of succeeding in external 
examinations. Surely this is an example of effective 
teaching? 

2. There are increasing numbers of students continuing 
their education in establishments of Further Education 
and Higher Education. 

3. In addition, adults follow courses of their own choice 
in adult education centres. 
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As far as the role of the computer is concerned we should 
ask: What are we trying to do in education? What are the 
resources? How best can we use them? 

If the computer is a resource— and I believe that it is— we 
should see that its use relates to trends in education. Our 
horizons should not be limited to considering CAI only. 

There are, for example, trends towards more independent 
and group work, to the use of learning modules-CMI would 
therefore be valuable in providing a means of assessment 
and diagnosis. Investigation in this field should be linked 
with current curriculum development projects. 

We should also examine the tool use of the computer and 
simulations. These offer a powerful potential for curriculum 
innovation, since they are likely to encourage us to review 
approaches to learning and to extend the learner’s experience 
beyond conventional bounds. 

Mr. GAf. Hodgson noted that, “The economic arguments 
put forward so far in this symposium seem to ignore the 
global pressures which are emerging for large-scale action in 
automation aid for education.” Phillip Coombs, Director of 
the Institute for Educational Planning (a UNESCO body) in 
his recent book ‘The World Educational Crisis” (Oxford 
University Press, New York 1968) points out that current 
educational systems will demand a most alarming proportion 
of national income. Only if some major programme of 
cost-reduction based on automation aid is mounted urgently 
will economic breakdown be avoided. 

1 do not agree that the present educational system is a 
failure. In this 1 agree with Miss Ash. But I do believe that 
all the trends implicit in long range social and technological 
forecasting make the present system obsolescent. If these 
points are to be entertained, then the thought must also be 
entertained that considerable national and international 
resources should be channelled into avoiding the anticipated 
crisis. Parochial economic justification becomes secondary. 

Dr. Seidel replied to Miss Ash noting that if she were 
serious then we have been pursuing ambiguous goals up to 
now. We are in an educational crisis in that we are lagging 
behind by a generation. The problems we see are. those of 
20 years ago, not today. We are trying to apply educational 
techniques which we should have used 20 years ago. 

Mr. F. Duerdert gave an industrial viewpoint of the 
economic problems of CAI development- 

It seems to me that what is sauce for the student should 
also be sauce for the master— in other words the principles 
found effective within the CAI system should be effective in 
a system for learning about CAI. Professor Pask showed 
yesterday that control was not necessarily authoritarian but 
could be co-operative or catalytic; also that there was 
advantage in the student’s choosing his own learning path 
to a declared objective. I believe that the best managers and 
“controllers” are primarily enablers, at least in an activity 
with any degree of intellectual content. The problem inherent 
in Dr. Seidel’s Systems cycle lies in getting past point one. 

If you can state the need, and have chosen the people with 
motivation and capacity, then control should be pretty 
minimal (from an authoritarian point of view). The difficulty, 
which 1 think is beginning to be realized about organization 
for controlling systems containing people, is that you have 
a pretty primitive controller (the management system) 
operating on a network of very complex devices (the human 
brains). Awaiting the outcome of “catalytic control” may 



be nerve-racking, but it is the only means to real progress. 

In considering the strategic area, Dr. Seidel looks at the 
possibilities of govemment/industry partnership. Although 
1 believe that we in the UJC. can profit from the considerable 
work done on CAI in the U.S.A., this may not be equally 
true when considering the strategy of partnership, because 
the two countries start from opposite sides of what may be 
the optimum amount of government intervention. It is worthy 
of mention that Industry is more co-ordinated in the U.K. 
than page 14 of Dr. Seidel’s paper suggests, because of the 
existence of the Industrial Council for Educational and 
Training Technology. 

Industry is in a simpler position with regard to defining 
its needs, which are less complex than those of Education, 
bat can be stated in such a way as to bring out the general 
benefit to the community. Industry enters into the model of 
this educational process (a) as a user of human resources and 
(b) as a developer and supplier of technological equipment 
and systems. The benefits it can hope to derive from (a), in 
what I think if the order of priority, are: 

(i) improved competence and motivation of existing staff; 

(ii) reduced wastage of trained people, i.e. rapid retraining 
to counter redundancy and technical obsolescence; 

(iii) earlier competence for new staff. 

Under the (b) heading the industrial concern is likely to be 
for effective investment. As Dr. Seidel has noted, Industry’s 
record of development spending stands up to comparison; 
but there is a problem of time scale. A million pounds spent 
on development now requires a resultant 13 million in sales 
in 3 years’ time, but 26 million if the sales do not appear for 
10 years— assuming in each case that 10% of the value of sales 
is available for repayment of costs of development and also 
that the funds cost 10% p.a. for as long as they remain 
unrepaid. It is interesting to notice that a development 
program costing £1M total spread over 10 years requires 
sales of £26M if spread over 10 years also-this is true 
whatever the shape of the build-up of development spend 
and of sales provided they have the same shape. A smaller 
sales volume will suffice to recoup, if the sales build up more 
rapidly than did the development. If development cost 
build-up is linear over a 10-year period, only about 10% of 
the sales is required to pay for the first 3 years’ work. 

An important requirement emerges, if the total programme 
is to place the minimum drain on resources, for very sound 
judgment in the earlier (possibly study) phases before the 
heavy expenditure has been committed. The greatest 
importance must however be attached to the need for a 
rapid market build-up at the end of the development 
period, and to attain this we need a well-considered public 
relations activity (as referred to in the NCET Study Team’s 
report), demonstration facilities, and better organization of 
the whole educational market which currently combines 
the worst features of the systematic vagaries of central 
funding, with the randomness and high costs of individualized 
selling. If adequate and early attention is given to the market 
problem, I do not think that Industry will be lacking in doing 
its part in the development programme. 

Dr. L.C. Jesty reported work on “Electronic Aided 
Instruction” at Chelsea College (University of London), 

“The choice of this title is deliberate as, although we are 
using a computer, it is a component in tire system and the 



operation is not dominated by it. in addition to its technical 
accuracy therefore, the word ‘Electronic also gives the 
correct emphasis to the direction of our efforts.” 

The investigation began with a preliminary survey of the 
work already underway or planned in the U.K. in' this field 
of Instructional Technology, (a) to ensure that we did not 
indulge in unnecessary duplication and (b) to direct our 
own efforts most effectively. The present programme is 
sponsored by the Electronics Dept, of the College, under 
Prof. Houldin, and is being carried out in collaboration with 
the Centre for Science Education, under Prof. Keohane. 

The primary objective is to investigate the possibility of 
settingup standards for a two-way communication channel 
between a Central Programme (Lesson) Distribution Centre 
and suitable student terminals which will give worthwhile 
help in dealing with the educational explosion which has 
already arrived. We consider this objective must be achieved 
inexpensively. It is not so much a question of “what can we 
provide” as “what can we afford”. If the channel standards 
envisaged could cater for the education of 90 % of the pupils 
for 90% of the time it would be ample justification of the 
project but a much smaller proportion would still make it 
worthwhile. 

There are many parallels to be found between the present 
state of educational technology and television in the early 
’30’s. At that time television “Systems" were prolific; 
equipment was inadequate; programmes we. e non-existent 
and so was finance. The establishment of sensible standards 
and the inauguration of the Television Service in 1936 
immediately put the pressures in the right places. 

We are at present assembling equipment, and the first 
exploratory experiments will be carried out using simulation 
wherever necessary. A parallel engineering investigation will 
evaluate the possibility of providing the electronic and other 
equipment to replace the simulators, and if necessary, modify 
the standards. As soon as the simulation equipment is 
operational we shall attempt to deal with as wide a variety 
of programme (lesson) material as those qualified to do so 
can provide. We shall welcome suggestions and material at 
the phase of the work. 

It is hoped that the two-way channel envisaged will 
eventually be capable of dealing with all forms of teaching 
technology ranging from a simple one-way closed circuit 
television channel conveying the same lesson simultaneously 
to a large group of pupils, through all the intermediate 
stages of branching programmes with student response and 
evaluation, to the fully-conversational computer based 
learning situation giving individual tutorial instruction. The 
same student terminals will be used throughout. The only 
difference between these limits will then be the cost per 
student-hour in operating the system. The proportion of 
the central distribution equipment allocated to each student 
will increase with the complexity of the lesson strategy and 
with the degree of individual tuition involved. If our objective 
can be achieved-and this we have to determine-then the use 
of the channel can be regulated entirely on the basis of the 
finances available to the operating authority. 

Mr. J. Duke enquired about the possibility of educational 
technology at the level of CAI which did not necessarily 
involve a computer. 

Prof. P&sk replied, “Yes, there are at least two relevant 
activities that do not necessarily involve a computer, though 



in each case a computer might be employed i f it turned out 
to afford a less expensive or more convenient tool than 
special-purpose equipment.” These two activities are: 

1 . Research and Development- Especially research into 
educational subroutines (in the sense of my paper). A great 
deal of my own work, and that of others, has been done 
using special-purpose equipment. 

Quite elaborate man machine interfaces may be required 
for studies of control strategies (in an educational subroutine) 
since, over and above the task in hand, the student must 
receive: 

1. various (mechanized) instructions, and 

2. evaluations and he must respond with (mechanized) 
statements and selections. 

Whereas it would be easy to instrument the required facilities 
with one of the elaborate CAI interfaces (C.R. Tube, projector 
and so on) this -expedient is only mandatory if string 
manipulation is required. Otherwise, it is probably no more 
expensive to build an interface for each experiment. 

So far as computation and recording is concerned, some 
special-purpose equipment is needed in the absence of a 
computer but the experimenter, working from a flow chart, 
can do most of the deep computation, parametric adjustment 
and so on providing he is given summary performance data 
and provided the serious business of recording is off-loaded 
onto a mechanical system. Apart from giving the initial 
instructions the experimenter does not, of course, interact 
directly with the student. Clearly the experimenter and 
some of his equipment should be replaced by a small 
computing machine and in that case it wouid be sensible to 
computerize the data recording as well (at present we 
process the tapes after the experiment and only do summary 
computations during the experiment). To do so, would call 
for the commitment of one small machine to each student 
(say a PDP 8 with 4K additional storage) or its equivalent. 

2. Operating Systems- The real merit of research in CAI is 
that it provides (for programming, it must provide) a clear 
statement of how instruction should proceed. It is a matter 
of economy, convenience, and the need for personal contact 
that determines whether (1) the procedure should be carried 
out by a real life teacher or (2) by a CAI system. Nor are 
(1) and (2) exclusive. In many systems a good solution can 
be achieved by mixing human and machine administration. 

A couple of “hybrid” systems have been developed in my 
own laboratory, namely COMOPTS (for COMCEN operator 
training) and SIMPOL (which is being fitted with a TUTOR 
supervisor) for managerial training of detective inspectors. 
Both of these systems use the human teacher in two different 
ways: (a) as someone who instruments a procedure assisted 
by summary data and interface computing apparatus and 

(b) as a private instructor who is called upon from time to 
time by the procedure and who, in that role, interacts 
directly with the student. Obviously, function (a) can be 
computerized at will and it may or may not be economic to 
take up this option. In general, function (b) must remain in 
human hands. 

Dr. Karl Zinn stated that it was not definite that the 
computer would have an ultimate role in any educational 
application where it is currently used for research. There 
are motivational techniques of equal sophistication to CAI 
where often the goai is to get people to interact with other 





people. He illustrated this with the example of Layman 
Allen’s academic games, not using computers. Originally 
these were to be designed for CAI, but in analyzing them it 
became clear how to implement them without a computer. 
Finally, a computer was used only to generate tables which 
were put in booklets. 

Dr. Derek Sleeman noted that the computer was a tool 
for use in a deep and thorough analysis of the educational 
system. The tool could be misused and if rubbish was put in 



then rubbish came out. He felt that it made sense to use a 
small computer system as a CAI testbed, going over to large 
systems for implementation. 

The remaining discussion covered some of the internal 
problems of the IBM 1500 system, the importance of 
students as programmers in software development, and the 
relative costs of computer hardware and the manpower 
necessary to drive it. 
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Introduction 

By the time this paper is considered at the Seminar on 
Computer Based Learning Systems there will have been 
considerable discussion on whether, or not, there is a need 
for such systems in the United Kingdom, whether such 
systems can work in the United Kingdom and on whether 
the United Kingdom should mount any kind of National 
Programme of Research and Development into their use. 
Since so many of you, I know, are to a greater, or less, 
extent, supporters of such systems, I will assume that you 
have, by this time, agreed that there should be a National 
Programme. My task, therefore, is to try and "«t out some 
kind of proposals on how such a programme s >uld be 
planned and managed. In attempting to carry i.us out I 
shall draw both from the experience I gained as member of 
the study team which NCET set up to examine the use of 
computers in education and as a member of the Mintech/ 
UKAEA Programmes Analysis Unit, which was set up some 
two and a half years ago to evaluate R&D proposals which 
reqr'.ed the support of the Ministry of Technology. I would 
emphases, however, that the views expressed in this paper 
are not necessarily shared by my colleagues on the study 
team or in Programmes Analysis Unit. 

The American Scene 

As you all know, the study team started off its consideration 
of what should be done by way of using computers in 
education by making a visit to the United States of America. 
This served to provide the Team with first hand e tperience 
of the new technologies in use in the United Stales of 
America and a report of our visit has been written up. I 
expect most of you will have seen it by now. As well as 
looking at the various technological and educatioral aspects 
of this small sector of the American scene. I found it of 
great interest to try and find out how the various projects 
were managed and how the results of the various studies 
were brought together. It was also of interest to me to try 
and determine how they were funded and to find out how 
this affected the overall plan. My objective in doing these 
things was not only because of my personal interest in the 
administration of research and development but because it 
would be valuable to draw on USA experience if a similar 
programme materialised in the United Kingdom. 

I do not propose (and indeed I would not be capable of doing 
so) to provide you with a historical account of how the 
various studies on computer based learning systems grew up 
in the United States of America. All I can do is to give you 
my highly personalized view of what I saw in the Autumn of 
1968. 

Some American Projects 

During the four and a half weeks which the team spent in the 




United States of America visits were made to over 40 projects. 
The overall impression I gained was of visiting over 40 
different projects. Some of these were small, while others 
were very large, both in terms of objective and cost. Some 
looked as if they would be successful, others looked as 
though they were staggering on to their end. There seemed 
little correlation between the two factors of size/cost and 
probability of success. This, however, is not a phenomenon 
unknown in this country nor is it confined to the field of 
educational technology. All of these projects rejoiced in the 
title of research projects but to my mind really formed 
parts of the banded spectrum of pre-production work 
labelled R&D. 

This spectrum of work was divided into three main parts. 
Firstly, a large amount of it was, by and large, of a ‘basic 
research’ character. By this I mean that a topic- and a method 
of attack is determined by a particular individual to satisfy 
his own tastes and intellectual curiosity, rather than because 
a ‘need’ has been identified or that the work will lead to a 
discovery which will be of any practical use. Much of the 
work we saw was indeed valid ‘basic research’ in that it was 
carrying out semi-empirical experimentation in order to 
identify some real-life problems. It is often neither desirable, 
nor indeed possible, to ‘plan’ such research, except perhaps 
to put some upper limit to expenditure before reviewing 
the next stage. The decision on whether this is done, or not, 
seems to rest with the particular funding agency and 
practices clearly differed. 

Secondly, a considerable sector of the work being carried 
out was not of this ‘marginal basic research’ but was more 
‘applied research’. It covered research in fields of recognised 
potential technological importance (I would define the 1 
imparting of knowledge as a c orm of technology). This type 
of research demands a more planned approach in that there 
is, or should be, some clearly defined technical objective to 
aim at. For example, in some of the projects we visited the 
objective appeared to be to find yet another technique of 
using the computer without, however, much thought being 
given to either need or desirability. Despite the fairly clear 
objectives of this type of work there was little evidence 
which I saw of any attempt being made to direct the 
research towards that objective or to resist temptations to 
be side tracked. Indeed this work was just as unplanned 
and uncontrolled as the basic research work. 

The third major area of work going on was very much in 
the applied research and development field. Since this work 
was largely under industrial control, there was evidence of 
a planned, and perhaps too tightly controlled approach. 




The Research Approach 

It is, I think, of interest to try and find out why the relatively 
unplanned tackling of a project of this kind has developed 
and to try and determine the advantages and disadvantages 
of such a method. There is no doubt that a great deal of the 
work we saw was started up mainly in Universities because 
of the interest and curiosity of some individuals. Seldom did 
a request for aid come from practising teachers or educa- 
tionalists. Depending on who the individual was, his interest 
would be for example, in learning theory, in human 
behaviour, on using a computer or perhaps even in selling a 
computer. Again, depending on his interest, he obtained 
funds from one (or more) of a number of different funding 
agencies. To judge from their titles these funding agencies 
covered a wide range of interests and it was only by some 
stretching of the imagination and some fairly close question- 
ing that it was possible to determine why a particular 
agency was funding a particular piece of research. Included 
among the more easily identified sources of money were the 
United States Department of Defense, the Office of Naval 
Research, the Office of Education, Office of Economic 
Opportunity, National Science Foundation. It was difficult, 
for example, to reconcile the interest shown by ONR in 
school education with the funding by OE of the U.S. 

Navy project at Annapolis. Others, more difficult to 
identify, although their interests were clearer, were the 
local and State School Boards and internal University 
projects. Another source of funds, both for internal and 
for extramural activity, were the large industrial companies 
such as IBM, RCA, etc. Undoubtedly, support was also 
obtainable from organisations such as the Ford Foundation. 
This technique of funding seems to be quite common in 
the United States of America. There are no doubt historical 
reasons for it and that it continues is probably due to the 
active part played by various pressure groups. It is most 
refreshing even if somewhat confusing, to find that these 
‘Establishment’ type organisations refuse to be ham-strung 
by their own red tape and are not constrained by ‘teims- 
of-reference’. I wonder what effect this has, however, on 
the ‘not invented here’ complex when it becomes desirable 
to use the results. Would the Office of Education be 
prepared to support an ‘applied project’, based on the 
results of ‘basic research’ which had been funded by the 
Department of Defense? I think I know what the answer 
would be in this country. 

It is not surprising, therefore, to be faced with a bewildering 
assortment of projects, the proposals for wliich had had to 
be framed to take account, not only of the personal 
interests of its initiator, but couched to make the maximum 
effect on the particular source of funds chosen. 

After the funds have been granted the personal interest of 
the protagonist seemed to take control again, and so the 
aims of the project, in most cases, tend to deviate from the 
objectives given in the proposal. The effect on the observer, 
such as I was, was rather devastating. I would be told by the 
sponsoring body that Dr. X at Y University was engaged on 
a particular study but on visiting Dr. X at Y I would find a 
completely different study going on. Some of the apparently 
successful practitioners of this type of R&D seem to have 
made more a study of where to receive their funding than 
they had of trying to identify the solution to their problem. 



free publication can be produced. Many of these are valid 
but perhaps because of the ‘publications explosion’ many 
of the people we saw were not aware of what others were 
doing in a related field. In fact, one of the first questions 
Team Members were asked on practically every visit was 
‘can you tell us what you saw in . . .?’ There are other 
explanations for this lack of knowledge of parallel work 
which I don’t necessarily subscribe to but which cannot be 
entirely ignored. Is there a fear that someone might have 
shown that your own programme wasn’t necessary? Or is 
it that in order to continue to get funds the hypotheses are 
somehow presented as facts? Or is it that people don’t 
publish enough of the right information? Whatever the 
reason there was what to my mind was an appalling lack of 
information about similar work, not only at the other end 
of the country, but often in the same, or neighbouring, 
State and even in one case on the same campus, indeed 
within the same department of the same campus. 

Is it any wonder then that the overall picture appears to be 
such a hotch-potch and that there seemed so little to show 
for 4/5 years’ effort at an annual expenditure of many 
millions of dollars per year? 



Applied Research 

As I have said the relatively unplanned basic research type 
approach can be justified for some of the work we saw 
carried out in the United States of America. Many of the 
projects we visited had, however, gone beyond this stage 
and were more concerned with trying to solve the problems 
of using the results of the basic work. We found, therefore, 
projects which were attempting to produce CAI/CMI 
techniques, which could be used to teach some specific 
skill or topic. We saw many projects where such techniques 
were being used in a teaching situation. Many of these had 
progressed beyond the ad-hoc experimental phase and it 
was reasonable to assume that someone had identified a 
reason for doing them. It is, to my mind, much easier to 
spot the disadvantages of current United States practice 
in this area than to pinpoint any advantages. 

One major disadvantage is that if there has been no clear-cut 
objective to aim at it seems to me very difficult, if not 
impossible, to assess how close, or how far off, you are to 
the target. How then can the success, or otherwise, of the 
project be determined? To be told as we were on many an 
occasion that CAI was ‘better’ than classroom teaching 
required the question to be asked (and we often did) ‘How 
do you know?’ Unfortunately, this question was nearly 
always unanswered. Perhaps the objective wasn’t to show 
CAI was belter than ... If so, what is the objective? Again, 
we were baffled. However, in the U.S A. context our Team 
was of no significance, and rightly so! However, there was 
just an inkling that some of the funding agencies, particularly 
those providing direct Federal support, were beginning to ask 
the same questions. Unless some better answers can be 
produced than we got then there seems to be real danger of 
a drastic cut in funds for this whole project. Let me say here 
that I do not think it always to be necessary to show that the 
‘new’ approach is ‘better’ than the old one. There are many 



, * mus- a. r ii»>ii! wiiiiiviiL * ji wi w 



There is no doubt that a n approach such as this can be 
attractive in that it enables a wide spectrum of problems to 
be examined by a large number of people of widely 
different interests and abilities. It is, however, expensive 
both in real money terms and in manpower and can only 
begin to be justified when truly b*sic research is being 
carried out. 

On the question of communication between projects all 
the arguments relevant to the benefits and advantages of 
cases where the ‘new’ system does what the old one cannot 
do. Or a completely new vista can be opened up for studenis. 
If, however, protagonists of the ‘new’ system claim it to be 
‘better’ then they are responsible for making this comparison 
and surely must attempt to justify their claims. Equally, 
however, I cannot accept that just because a method is ‘new’ 
then it must be ‘better’. 

In many of these ‘applied research projects’ we sensed that 
the people working on them felt they should still be doing 
‘basic research’ and were using the rather ‘airy-fairy’ type of 
approach which seems to go with this type of work. It 
seemed that some of the funding agencies, particularly those 
under close Governmental control, were tightening up on 
their ‘basic research’ budgets and the only way these projects 
could continue was for them to be aimed at solving particular 
problems. No doubt there had been some very skilful re- 
writing of proposals to suit the needs of the funding body 
but this seemed to have been done so quickly that insufficient 
time had been given for the workers in the field to readjust 
either their objectives or their methods of working. In some 
cases it was clearly wrong to expect the basic researcher to 
continue with the new type of programme. It was not clear 
to me whether it was only the project staff who were paying 
only lip sendee to their objectives, or whether, in fact, the 
fault lay much more with the funding agency, who were 
being forced to at least appear to be solving problems. 

Another of the signs that there is no co-ordination in the 
present system is the use of so many different high level 
languages. Referring to my personal notes of my visit to the 
United States of America, I came across references to 
between fifteen and twenty. This, I may add, was not an 
exhaustive list. I have no doubt that each and every author 
could demonstrate at least to liis own satisfaction how 
impossible it was for him to use an existing language. 

Surely, however, one of the axioms of scientific work is 
that results should be verifiable by any other worker in the 
field. How can this be done at present? Allied to, or perhaps 
one of the reasons for, this state of affairs is the multiplicity 
of computers and computer systems. This, of course, 
impinges on the field of commerce and no doubt again can 
be justified. But I wonder what effect all this has on 
progress towards solving the problems of education, which 
CAI is supposed to deal with. Or is it? Has anyone 
attempted to identify what problems in education the 
computer is supposed to solve? We found very few teachers 
taking the initiative in the setting up of projects. Is it that 
they haven’t yet realised they have problems? Or is it they 
feel there are other ways of solving them? Again I think 
this points to the need of at least trying to identify an 
objective. 



Development 

This phase of the project involves designing the system and 
carrying out prototype operations. Again, this is not a clearly 
defined sector of the operation but is analogous to the band 
at the opposite end of a continuous spectrum from basic 
research. Here we found a number of projects were going on. 
Many of them invblved the use or development of specific 
items of hardware or software. Most of these were being 
carried out by, or were funded by, industrial companies 
particularly by computer companies. In these, as might be 
expected, there was evidence of fairly clearly defined aims 
and objectives. Sometimes these were couched in terms to 
allay the suspicions of the educationalist, but often they 
were perhaps more honestly directed towards the shareholder. 
In this latter case the crudest but most honest form was to 
aim to increase the sales of X’s products. 

Here again, but perhaps more understandably, there was 
little, or no, attempt at co-ordination of projects although 
in some cases an attempt had been made to carry out a 
wide ranging series of experiments using the same basic 
system. The example which immediately comes to mind 
is the number of projects using the IBM 1500 System. We 
were told, by some of the IBM people, however, that the 
results were less useful to them than they had hoped. What 
we were not clear about was whether these experiments 
had been initiated to assist IBM identify a market or to 
assist educationalists to try out new approaches to 
education. 

The Overall Position 

I have been, perhaps, unduly critical of what I saw in the 
United States of America? If so, I apologise to our guests 
from there. It might be that I expected too much and that 
having been told of the vast amounts spent over the past 
4-5 years on computer based learning systems I was 
expecting that problems where these systems could be 
used had been identified, and that planned experiments 
would have been undertaken to compare the worth of 
different systems according to some sort of criteria and 
that there would be some attempt to identify what still 
needed to be done. Unfortunately, I found very little of 
these had been done. Instead I found a bewildering amount 
of research-type projects, some good, some not so good, 
but all of them showing an altogether awe inspiring 
command of technology, particularly the technology of 
using existing computers in an on-line mode. This was 
very impressive but as has been said by someone else 
about a town famous for its part in another great techno- 
logical achievement, ‘everyone knows how, but no-one 
asks why?’ 

The Over Planned Approach 

Having examined the relatively unplanned approach to 
R&D exemplified by the computer based learning projects 
in the United States of America, we must now look at an 
almost diametrically opposed system of carrying out such 
a programme. In this method an attempt is made to define 
clearly the aims and objectives of each small sector of the 
project as well as of the whole project. Estimates are made 
of the resources in manpower and money likely to be 
required and of the timescale which has to be met. 
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During the course of the project strict adherence to the time- 
table would be required and deviation from the planned 
programme would require careful explanation. 

The advantages of such a scheme are that the sponsors can 
be very sure of how and when resources are used. They can 
integrate this project with others to enable optimum use of 
those resources which are difficult to come by. This type of 
scheme brings joy to the accountant’s heart. It ensures that 
those scientists whom he believes to be a useless and feckless 
crew engaged on spending money for their own amusement 
are brought under control. The scientists and technologists, 
however, being honest and tolerant men, can very easily 
point out why such a system of control would not work. 

They would claim that when new ground is being explored 
objectives cannot be clear, that precise programmes are 
dangerous because the most valuable discoveries are 
unexpected ones and that genius must not be cramped by 
budgeting. 

Both parties to this kind of argument could produce 
examples of how right their approach is. They will continue 
to do so and money and resources will continue to be 
wasted in many projects dealt with in both ways until there 
is more realisation that there are differences between basic 
research, applied research and development. The differences 
are not clear cut but because they are different activities the 
method of managing them and controlling them must be 
different. 

I would reject completely any proposal to conduct all R&D 
on a completely planned basis just as vigorously as I would 
reject any proposal to conduct it on the completely 
unplanned system which seems to be the delight of the 
basic and pseudo-basic research workers. 

The United Kingdom Scene 

There is one characteristic of the United Kingdom attitude 
to science and technology about which I think we can be 
absolutely certain. It is that neither money nor resources 
will be available to match the efforts of the United States 
of America. Having said this I think we must then ask 
whether in this particular field (e.g. computer based learning) 
we want, or indeed need, to match them. Can we learn any- 
thing from the work done there and elsewhere to provide us 
with a starting point? (The more fundamental question of do 
we need, or want to do anything at all has, I presume, been 
answered earlier in the week.) We have, in our earlier 
discussion this week, and the study team in their report to 
NCET (i), tried to identify the areas of activity which needed 
to be covered by a national programme in the United 
Kingdom. We must now consider how this affects the 
planning and management of the programme. 

CAI 

In the strictly CAI field we feel there is a need to apply 
some of the known concepts of learning theory and the 
known techniques of computer programming to the task 
of preparing teaching programmes of different kinds and 
in different subjects. The team recognised that a great deal 
still needed to be discovered about the mental processes 
which enable us to Team’ but felt (perhaps rather unkindly) 

O that as the education process had gone on more or less 



successfully for seyptal hundreds of years without all these 
problems having been solved, perhaps progress could con- 
tinue without waiting for their solution now. A partial 
analogy can be drdwn with agriculture. It will, undoubtedly, 
be possible to grow ‘better’ crops once we have understood 
all about the sciences of botany, plant nutrition ecology etc. 
but farmers do not, and cannot, wait until all the problems 
of these sciences have been solved before using empirical or 
pragmatic means to improve their products. It seems to me, 
therefore, that while it is of vital importance to improve our 
knowledge of how people learn, how people think, how to 
devise esoteric computer programming techniques and so on, 
the main thing we should be trying to do in the United 
Kingdom at this time is to harvest the fruits of the research 
already done. 

CM! 

When we consider the possible requirements for R&D into 
CMI then the story is very much the same. We have to 
decide what parts of the education administration system 
as it is now, and as we think it will be in the future, require 
the use of computers. Will this be in organising the subjects, 
the courses and the examinations of individual students? 

Will it be to assist in estimating fi’ture requirements in the 
way of schools, universities, teachers and so on? Will it be 
in dealing with the provision of information from libraries 
and data banks for students, teachers or administrators? 
Very few of these topics require basic research but all of 
them require considerable effort on finding out how to 
interpret the results already available from research projects 
and of how to apply these results to the foreseeable needs 
of this country. 

United Kingdom Requirements 

You will see, therefore, that while acknowledging the need 
for basic research into many aspects of the education 
process, we feel, and I consider rightly so*, that the main 
effort from the national point of view must be directed 
towards finding out whether, or not, the research alreadv 
done could help with our future educational problems. . 
while investigating these problems, we come across 
applications which would help to solve some of our current 
problems, this would be of considerable value. In all of 
this we recognised the likelihood of identifying the need 
for new basic research. We recognised that the results of 
current basic research might make it necessary to change 
course but we were, I think, completely united in believing 
that if a United Kingdom national programme was to be 
mounted now it could not rest on proposals for more basic 
research. The United Kingdom has often been accused of 
being good at basic research but neglectful of the applica- 
tions. In this case let us be sure that we can apply what is 
already known albeit by applying the results of someone 
else’s basic research. 



Having said this I must point out that very strong arguments 
can be produced for not mounting even a modest United 
Kingdom national programme now. This possibility exercised 
the minds of the study team for a considerable time before 
we rejected it. I do not propose to repeat these arguments 
here as they are set out in Chapter VIII of the study team 
report to NCET . 1 
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In summary, therefore, I believe that a potential need e j, ists 
in the United Kingdom for a learning system involving the 
use of computers. The immediate objective of a national 
programme must be to see whether these needs can be 
satisfied by applying currently available knowledge. If not, 
then the programme should help to determine what areas 
of knowledge still need to be explored. Let me repeat that 
in basic research the scientist does not know what he is 
going to find, he only knows the direction in which he is 
looking. In applied research he knows what he wants and is 
trying to discover how near he can get to it. If he cannot get 
close enough to justify development he might at least have 
helped to put up signposts for the basic researcher. 

The United Kingdom Programme & Its Management 
I have until now considered some of the more philosophical 
aspects of planning an R&D programme such as one on 
computer based learning systems, in doing so I have used 
my admittedly limited experience of the United States 
programme in this field to highlight the problems of 
applying the ‘basic research’ type of approach and to 
argue that the United Kingdom need is for ‘applied research’ 
and that this involves some form of overall planning. 

We must now examine how the United Kingdom programme 
is likely to be financed as this will have a very direct 
influence on how it has to be planned and managed. Here 
again there are major differences from the United States of 
America. There are few, if any, industrial companies in the 
United Kingdom comparable with IBM or RCA, who are 
prepared, or able, to finance even the in-house expenditure 
of these companies on computer based learning systems, 
apart from their contributions directly to identifiable 
experiments ran by schools and universities or indirectly 
through non-earmarked contributions to universities and 
colleges. In the same way there are fewer sources of revenue 
from among the charitable foundations in this country while 
local education authorities are not noted for their contribu- 
tions to educational research. Indeed, the Department of 
Education & Science spends very little of its total budget 
on this kind of research and develQpment. 

It seems clear then that if a national R&D programme is to 
start the government will have to be persuaded to foot the 
bill. Whether this is done by a direct contribution for this 
purpose, or via a number of different department votes and 
ledger headings, will not, I think, circumvent the need to 
persuade the Treasury to agree to the expenditure. This is 
the kind of expenditure which the government should in 
fact support, at least in principle. It is work on an area of 
activity where the public sector will be the main consumer; 
it has an end objective which is essentially a social benefit 
and it is of such a size, in its requirement for resources, that 
no industrial organisation would, or could, finance it. 
However, in practice government support will only be forth- 
coming if a case can be made which shows why this project 
should be given approval, rather than other projects which 
would use the same resources. Jn other words, because of 
limitations on money and men it is necessary to estimate the 
benefits likely to accrue to the nation from R&D expendi- 
ture. These benefits can, of course, be in intangibles such 
as improvements in environment and ‘better’ education as 
well as in purely economic terms It must be recognised, 
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however, that until we know better how to deal with and 
compare social benefits, the economic ones are more easily 
thought about and compared. 

While such an approach to making a research proposal is 
anatnema to the ‘pure’ scientist, we must, these days, be 
prepared to face the facts of life. Many people will argue 
that only by allowing a’hundred plants to bloom will we 
ever have a garden. It is as well to remember thatmany 
plants, if not pruned and kept under control, will give you 
an unmanageable and unproductive mess, the only remedy 
for which is to ruthlessly cut them down. 

In my view any work done in the United Kingdom in the 
field of computer based learning systems must have as its 
overall objective the identification of where and how com- 
puters can be used in the education system of the United 
Kingdom. This will involve, for example, experiments in 
writing teaching programmes, in methods of presentation, 
in assembling and analysing information about individual 
performances and so on. In the early stages there will be 
very little basic research work required and very little 
development and design of equipment needed. The various 
sub-objectives can, and should, be modified as the work 
proceeds, but the aim of reaching the overall objective 
should be preserved. This will involve adopting a moving 
plan rather than a fixed one. 

While the above must constitute the major part of the 
programme, there will still be room for, and indeed a need 
for, basic research. The planning of this cannot be too 
rigid. There must be flexibility to allow for examination 
of new ideas. I do not believe, however, that all of this 
basic work should form part of the United Kingdom 
national programme. A large part of it is of a type that 
should be conducted under the normal cloak of basic 
research and should be going on with, or without, a 
national programme such as we are considering here. It 
should have to compete for funds with other basic research 
projects and be judged on the same basis as they are, how- 
ever that may be. If, however, one of these basic research 
projects looks as though it has potential value to the main 
project then the main project must be flexible enough to 
adopt it for its use in reaching the overall objective. It will 
then cease to be managed as a basic research activity but 
will become an integral part of the project. 

As will be clear from reading the report of the study team 
and the recommendations of the NCET steering committee, 
as well as listening to the discussions -lere this week, there 
are many ways in which a detailed programme could be put 
together in order to aim at the same overall objectives. I do 
not intend to repeat here the details of these alternatives. 

It is sufficient to point out that there is agreement on the 
areas which will need to be covered by the R&D work and 
the order in which it should be done. There are some 

differences of opinion between individuals on ‘exactly’ how 
things should be done, on what precise kind of hardware 
should be used, and on where different projects should be 
sited. It is to be hoped that one of the results of this 
Seminar will be to initiate even more detailed proposals 
for certain sectors of the work. None of these differences, 
however, prevents us carrying out the strategic planning of 



the project at this time. In my opinion we must do this in 
stages. I do not think that we can, with our present state 
of knowledge, carry out any form of cost-benefit and/or 
cost effectiveness study to give us'an estimate of the 
upper bounds of expenditure. We do not know for example 
the contribution made to our Gross National Product by 
changes in the education system. Derek Medford, in 
Chapter VI of the Study Team’s Report} has put forward 
a possible mathematical model for this situation; but I am 
confident that he would not claim this to be anything more 
than a first attempt, albeit a very good one, at trying at 
least to identify what the important parameters are. This 
is a field of activity which I am convinced must be given 
urgent attention. One essential part of the programme must 
be the identification of those aspects of the education pro- 
cess which contribute most to national progress and which 
will most readily lead to the achievement of the nation’s 
will. Another factor which prevents the carrying out of cost 
benefit/cost effectiveness studies is the difficulty of 
obtaining agreement from experts on the aims and objectives 
of education. Speaking for the technical economists in the 
study team, we would have welcomed, and indeed we pleaded 
for, some guidance on these. Nor surprisingly we were dis- 
appointed. We found ourselves unable to identify a ‘learning 
systems model’ (or even a teaching one!). This accounts, in 
part, for my lack of enthusiasm for basing the national 
programme on the results of basic research into learning 
theory or behavioural patterns etc. Let me repeat, these 
things are important and if someone can see how currently 
available knowledge can be used, then by all means let us 
use it, but it must be knowledge which is reasonably 
acceptable as truth. 

We return then to the problem of reconciling these uncer- 
tainties into a coherent proposal. The first question asked 
by the Treasury in these matters is usually ‘how much?’ 
Although we cannot be precise at this stage, there does 
seem to be fairly broad agreement that over the first three 
to five years a total expenditure of about £2M would be 
necessary to achieve anything worthwhile. I would emphasise 
that this sum includes the various overheads etc., which 
universities in particular feel unashamed in ignoring. It 
also excludes any benefits likely to arise from using the 
computer to save other expenditure. Such savings have a 
knack of never materialising. Assuming this sum comes 
within the upper limit of expenditure which could be 
countenanced, we must try to identify some form of 
organisation to ensure that the money is well spent. 

The Board of Management 

Assuming, as I think we must do at this time, that finance 
will be by the Treasury through a large number of individual 
department budgets, there will, of necessity, be a group of 
people representing all these departmental interests. It would 
be nice to think that they would also all be representing the 
interests of achieving the aims of the project. These people, 
part-timers, would constitute a board of management. They 
are not likely to be in q position of taking any direct execu- 
tive action within the project, as they would have neither 
the time nor presumably the inclination to take the necessary 
day-to-day interest which executive control involves. 

The Directorate 



Serving this Board would be a small full-time directorate. 

This would consist of three or four people, whose essential 
qualities would be the ability to oversee a number of projects 
and to withstand the pressures from the more'voluble 
sectional interests involved. These people would, in other 
words, be good project managers. They would not need to 
be experts, either in computing or educational psychology 
or teaching or in any of the subjects being considered for 
CAI techniques. They will have the responsibility of firstly 
framing the detailed proposals and cf ensuring that the 
overall aims and objectives are adhered to within the overall 
allocation of resources. They must not hesitate to modify 
the detailed plans within these limits, or to propose additions 
and deletions to them. In fact an ability to say ‘stop’to a 
project director might be an essential feature of the job 
specification. The appointment of this Directorate is to my 
mind the key to the whole project. Only when these people 
have got together and have discussed among themselves the 
ideas for individual projects put forward not only by those 
currently' involved in this work, but also by those whose 
imagination has been kindled - perhaps at this Seminar it 
will be possible to sort out in detail a programme which 
looks like achieving the overall objective.* There will be 
much argument and discussion; much give and take before 
the project begins to take shape. Only then can firm 
proposals be assembled. 

Projects 

It seems clear to me that in order to obtain the maximum 
value from the national programme, as outlined, it will be 
necessary to carry out work at a number of geographically 
separate centres. One important requirement of the organi- 
sation for the project is that these different centres can, and 
do, communicate with each other and that there should be 
the opportunity to cross check results obtained at the 
different centres. This will require early agreement on 
methods to be used and to some extent on the hardware 
and software to be employed. It would not necessarily be 
either sensible, or indeed possible, to start off by insisting 
that only one type of computing system should be used 
but it would be sensible to tty to ensure that only one, or at 
most two, competing languages should be used. It will also be 
essential to ensu ~ the maximum amount of compatibility 
between the systems in use. 

It has been said that ‘the total cost of unnecessary computer 
language differences may well be of the order of hundreds of 
millions of dollars annually’. Although we have not, as yet, 
reached that state in the United Kingdom, we are fast 
approaching it. It might be that one of the by-products of 
this Study might be to prevent this situation arising. This 
will require both good judgment and firmness from the 
project management to achieve this without stultifying 
the ingenuity of the workers. 

The Project Directors 

As I see it then, the national programme will be built up 
from a number of centres, each of which will have a 
reasonably clearly defined role, both for itself and for its 
part in the overall programme. Each centre will have a 
project director, who will be responsible for ensuring that 
the centre works towards these objectives. At each centre 
will be multi-discipline teams of computer specialists, 



subject specialists, teachers, psychologists and systems 
specialists. 

The key role at each centre will be played by the project 
director, who will be required to keep the correct balance 
between the conflicting demands of the various specialist 
groups. Each project director will have, of course, con- 
siderable freedom o? action within his own centre and 
within his own budget. They will report to, and be 
responsible to, the board of management through the full- 
time directorate, who have to ensure that each centre is 
working towards the overall objectives set for the project. 

J* is not possible to be specific about the organisation of 
each centre. This will depend on the programme tor that 
centre, but in all cases it will be desirable for some form of 
service contract to be used to ensure that the staff s day- 
to-day loyalties are towards the project rather than to their 
parent organisation. This will be particularly important in 
the case of part-time staff or staff on loan or attachment. 

It is not suggested that such staff should cut off all ties 
with their parent organisation. This would neither be 
practicable nor desirable. Indeed, the maintenance of 
their particular expertise over a period of two, three or 
four years will depend on their maintaining contact at the 
correct working level. 



Conclusion 

There are many facets of the planning and management of 
a national R&D programme on computer based learning 
systems which I have not discussed in this paper. This does 
not mean that they are unimportant; many are of vital 
importance, for example, how important wi'l it be to have 
the ‘right names’ associated with board of management? 
Should we have a Lord Robens, or a Lord Beeching 
there? Or is it more important to have the right people 
there and in the directorate and the projects? How 
important is the timing of this? I know some of you have, 
what I consider to be, very strong views on this, which I 
hope you will express. I do not claim to have been either 
exhaustive, or indeed profound. I feel that there is a need 
for an R&D programme (with emphasis about left of 
centre), that such a programme can, and should, be mounted 
now but that the maximum benefit can, and will, only be 
obtained from it if it is planned and managed as a ‘mission- 
orientated applied research’ programme. It is with this in 
mind that my proposals have taken shape. 
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DISCUSSION 



Mr. McLaren in introducing his paper said that the views he 
expressed were his own and must- not be attributed to all or 
any of the members of the NCET study team, or to the 
Programmes Analysis Unit. 

The different proposals put forward for the scope and 
content of an R&D programme do not affcet greatly my 
point of view on how such a programme should be managed. 

I will admit that a lot of consideration should be given to the 
proposals that effort should be devoted to investigating how 
people learn, think, and respond. 

This week we’ve heard a lot about learning theories, but I’m 
not convinced that they’re very different from the ones that 
we had tv/enty-odd years ago, and I do not think it helps us 
very much along the way to doing the job which the Study 
Team was asked to do — and that was to examine the feas- 
ibility of carrying out a large-scale R&D national programme 
in the U.K. on computer-based learning systems. The con- 
clusion I came to after my American visit, after reading a 
lot of reports, and talking to a large number of people, is 
that if you want to say at this stage to me, ‘Should we have 
a computer-aided instruction system in the United Kingdom 
at the moment?’, all the things I have heard, all the people I 
have spoken to, all the things I have seen, would not 
convince me that we should — because I do not think we 
know nearly enough about it. However, I do not think 
concentrating entirely on a programme of looking entirely 
at the psychological and behavioural aspects, even carried 
on for twenty years, would bring us one whit nearer the 
answer at that time. But from what has already been done, I 
think we have reached a stage where a good case can be made 
for trying CAI to see what happens. The unfortunate thing is 
that such an experiment requires a certain minimum 
expenditure if it’s going to be viable at all. 

Unfortunately, that expenditure is very large compared to 
the usual expenditure on educational R&D in this country, 
so we have very carefully to examine not only whether we 
should do anything, but indeed how we do it. We constantly 
get told in this country that we are very good at doing 
research and not very good at applying.the results of it. We 
also have a tendency to modify this a little to the stage of 
saying that, not only are we very good at research, but 
perhaps nobody else is very good at research, and therefore 
if any new ideas come forward before we would even think 
of applying them, we must re-do all the research ourselves. 



come from the United States because I concluded that, 
perhaps more so than would have happened in this country, 
the authors were prepared to put in print a criticism of 
themselves. 

They very openly admitted that they have made mistakes in 
the past few years; and that is always the first stage in trying 
to make sure to do better next time. I think we should all 
learn from tliis. 

I hope I’ve said enough in my paper to demonstrate that I 
don’t believe that any great progress will be made through a 
large number of small-scale experimental set-ups. I think 
there has to be an attempt made to identify nationally what 
our objectives should be and to operate on a viable scale an 
experimental programme which will try to meet these 
objectives. 

This isn’t a problem just for computer-aided instruction. Last 
week I was attending a conference of production engineers 
who were talking about introducing computer-aided design 
and numerical controlled machine tools. There was an 
amazing similarity between the problems that are being faced 
there especially the decisions that have to be made on what 
needs to be done now, and the problems that have cropped 
up here this week. 

There was great emphasis put on the need in engineering to 
set up integrated teams of design engineers, production 
engineers, maintenance engineers, and of people with a 
concern for human beings. I think this is something the 
education system should perhaps take note of. Even hard- 
headed industrialists who in the past would never give any 
consideration to a person as a person, are now beginning 
to believe that one of the major problems in the industrial 
field is how to get people to cooperate. 

This is a thing which educationalists need to watch very 
carefully in the future so that they go off in the opposite 
direction when industry is beginning at long last tc realize 
the importance of human problems and values. 

The planning and management of a complex project in CAI 
depends largely on the concept of integrated teams. I think 
it’s inevitable that in each team a leader will appear. He may 
be appointed, he may just appear. This leader may be a 
computer scientist, a psychologist, a systems analyst, or a 
humble school teacher. I don’t think it matters all that much. 
What does matter is that that one individual must not exert 
too great an influence over the rest of the team. 
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What we have to do in this case is to learn the lessons from 
what has already been done and adapt the ideas, inventions 
and systems that have been developed - particularly in the 
United States — to our particular needs in this country and 
see whether they work. Now as well as learning lessons on 
the purely technical side of how to use the computer and 
how to apply it, there are some equally important lessons 
to be learned on how to run a system, or a programme. 

This is mainly what my paper is about. 

In writing my paper I did’nt want to upset some of the 
people in the U.S A. who had been very kind and helpful to 
us. I was somewhat relieved when I read the papers that had 



Further in overall control of a programme such as that 
proposed there must be a group of people, whom I’ve 
designated as the directorate, whose job it is to manage the 
programme and coordinate the individual projects. They 
should not concern themselves with being expert computer 
technicians or psychologists; they may work in arty one of 
the disciplines involved, or indeed be expert in none of 
them. The important thing is that they should be good 
managers. They should keep their eye on the objective, 
they should allocate the resources in the best possible way 
in order to make sure that that objective is reached in a 
minimum of time with a minimum of expenditure. They 



must also — and this is probably the most difficult task of 
all — do this constantly bearing in mind that in fields all 
round about them work will be progressing whether funded 
from this programme or from some other programme, and 
that people in different places will fcre producing new ideas, 
some of which it may be profitable to incorporate into the 
overall programme. 

It is at this point that 1 think the main difference must be 
mentioned between the Study Team’s proposal and the 
NCET’s final recommendation. At first sight they are very 
similar; they are about the same amount of money, they are 
about the same general topics, they are spread over about 
the same period of time. There is however one important 
difference. A Treasury official looking at the NCET 
programme might say, There are six different projects 
proposed which total £2 million; as a matter of principle, 
let us cut out a random selection of projects that will save 
us £500,000’. Now I believe that if this were done the 
programme could not achieve anything. 

However in the Study Team’s programme there is more 
flexibility, and if the Treasury came along and said you 
would only get one and a half million instead of two million, 

I still think you could possibly get by, because there is, 
while not duplication of effort, sufficient flexibility built 
in still to cover a really wide field in different places. Now 
this doesn’t mean that the budget has been artificially 
inflated to £2 million. I still think that two million is 
probably the minimum. 

However at this stage 1 don’t think the detail of what is 
going to be done exactly is important. I think the important 
thing is to be able to convince the powers that be - and this 
will require all the persuasion that can be mustered - that 
this is worth doing. Let’s face it — there is very little 
evidence that CAI will work in the context of the United 
Kingdom. There is very little evidence that it has worked 
anywhere. I believe that there are certain areas where it 
could work, where it could make a very valuable contribu- 
tion to education. But I don’t think we know nearly enough 
about it yet. We have to convey this act of faith to the people 
with the money, and at least convince them that if they give 
us the money, it will be used wisely. I think the first step is 
to persuade the Treasury that they should agree in principle 
and that then strenuous efforts be made to put together a 
very small group of people, probably the people who would 
constitute the directorate finally, to sort out from among 
the half-dozen or so proposals we have today a coherent 
programme. This isn’t going to be easy, because one of the 
great difficulties is in trying to distinguish between what 
has been done, what is being done, and what people hope 
will be done. 

The job of the directorate is going to be to probe very 
carefully, to persuade people to modify their own ideas to 
fit into an overall plan, and then to get this off the ground. 

I reckon this may involve a year’s work before any detailed 
proposals can be formulated at all. 

One of the great dangers is that whoever has this task will be 
forced into a position of making a detailed proposal before 
they’re ready for it, for there is a conflicting interest. This 
; s that there are a number of people who are already 



enthusiastic and a number of people whose enthusiasm is 
beginning to wane, largely because they feel, and rightly so, 
that they haven’t had the support that their enthusiasm 
deserves. Delay can be very serious. A large project takes off 
because of individual enthusiasms, and it’s a great pity if 
they’ve got to be held in check. 

This is the great dilemma for the sponsor of a programme. 

I’m not sure how you deal with it, because you obviously 
can’t spend money until you’ve got it and there is a danger 
that momentum will be lost. The philosophy I take is that 
we should be at the stage now of trying to identify why we 
want to do CAI in this country, and then direct our energies 
towards it. I hope the planning goes on, I hope a programme 
gets started. I have no personal axe to grind in this, but I 
would hate to see the programme founder because it was 
split up into too many little segments and sectors wliich 
couldn’t be demonstrated to be productive. 

Mr. Hubbard, chairing this session, remarked, ‘I think there’s 
a point to be learned from the history of some other develop- 
ments - particularly the development of atomic power. The 
thesis I want to illustrate is that it doesn’t really very much 
matter, if you have your major and ultimate objective clear, 
which route you take towards it, provided that you take a 
route which at the time you make the decision appears to be 
immediately practical, and you go for it as hard as you can. 

In about 1947 the British Atomic Energy Authority experts 
took the view that the practical system for a reactor was a 
gas-cooled reactor. The American Atomic Energy 
Commission took the view that the practical system was a 
water reactor. They belted ahead for twenty years and at 
the end of that time they both had effective working 
economic systems, and it would take today an extremely 
able economist to make a fraction of a penny difference 
between the operating costs. Now one might have said that 
one of them must be making a wrong decision, but in fact 
this sort of decision - whether course A or course B will 
take you to your objective — can be made provided you can 
see the first few steps ahead. I don’t think either party 
totally followed the course they’d planned for themselves, 
but it enabled them to start. So I suspect that the important 
thing is, having got your money, having seen what you can 
do that is.useful, you go ahead and do it. 

I’m very impressed with Peter McLaren’s paragraph in which 
he says that applied research and developmental work can 
point the way for the pure research worker. I tliink this is 
the way you really have to attack this sort of rather 
intractable problem - not by trying to see dearly where 
you’re going, seeing clearly what pure research you want 
done and getting that done first, but doing what is 
immediately available to you so that you can immediately 
get a psable , worthwhile result. This is terribly important 
from the point of view of getting continued support.’ 

Dr. Karl Zinn, as rapporteur, continued the discussion. 

‘I do not wish my remarks to be viewed as a formal, response 
on the topic of management. I am at best a specialist on 
programming languages for instructional use of computers 
and perhaps on instruction strategies. When I finally agreed 
to be rapporteur 1 decided to begin my presentation by 
listing the persons I had asked to fill this spot, indicating 
what each has accomplished in his position with govern- 



ment, non-profit corporations or universities, what his 
present problems are now, and in some cases why he could 
not be here today. However, rather than take time for 
anecdotes and generalities, I shall instead move to specifics 
of managing the R&D programme, at least providing a 
framework for comments and recommendations from other 
participants here today. 

I wish to encourage especially my colleagues from the other 
side of the Atlantic to continue their participation. Bob 
Seidel has already expressed his opinion about managing the 
R&D programme in his presentation and the associated 
discussion this morning, and he might now speak in the 
context of this afternoon’s topic. You already know of his 
role in a sizeable project for the U.S. Armed Services. 

Harold Mitzel has been unusually successful in funding 
education research and development at his institution. 

Duncan Hansen is running a large laboratory, and initially 
had to struggle to obtain grants and contracts by which to 
feed his 32-mouthed monster once the institution made a 
commitment to purchase the instructional system. 

Ed Adams has already expressed an important perspective 
for the subject, and his experience as a researcher in industry 
is quite relevant. 

Incidentally, I tend to agree with the critical remarks made 
by Peter McLaren and the ones in the longer reports of the 
Study Team’s visit to the U.S. I am quite impressed with the 
thoroughness and the thoughtfulness of NCET deliberations, 
and embarrassed that such a set of documents and critical 
view do not exist in the U.S. 

I would like first to provide an outline or framework for 
discussion of goals, and then move on to specific recom 
mendations. 

Mission-oriented research and development. If a large 
programme is established with a specific mission and co- 
ordinated planning it will take up most of the funds which 
might be available for research and development in the area. 
Care must be taken not to exclude from the overall national 
programme those basic research projects which could not by 
their nature fit a specific mission. One can say that funding 
an applied programme will not compete with sponsorship 
of research as has been done in the past. However, new 
research will have to be championed by the Directorate of 
any specific programme, since those holding the purse 
strings will repeatedly refer proposals, for basic research 
along with others, to the Directorate for handling, or at 
least for review. 

Programme goals, have listed three kinds of goals expressed 
or implied in the NCET writings. Perhaps some of you would 
prefer to call these to c.nents the means to achieve some 
more substantive goais, or phases in a sequence of develop- 
ment and implementation. Whatever the label, I invite 
comment on the relative importance of: 1) development of 
tools; 2) demonstration of technology; 3) diffusion and 
change. 

1) I have heard much talk about developing computer-based 
tools for computer learning and teaching, tools for which 
the potential uses have been identified in research already 
completed. These tools would include curriculum and the 
personnel to apply it, as well as the system, devices and 



past successes have been of such a trivial nature (in light of 
computer capabilities) that they provide inadequate basis 
for determining future directions of applied research, 
programming techniques associated with computer use. A 
crucial question which I find implicit in the NCET working 
materials concerns the transition from initial probes where 
progress was easy (e.g., drill in arithmetic or language 
vocabulary) to a coordinated effort to remove the obstruc- 
tions to achieving significantly greater rewards. Most of the 

A second question I have about development of tools 
concerns a split into: a) tools for information processing 
(used by students as well as instructors), and b) tools for 
systematic instructing (used by the instructor-author- 
manager to apply pre-defined learning strategies to 
individual students). The latter set of tools (systematic 
instruction) should be derived in association with other 
efforts in education, training and performance. Other 
systems of similar purpose and character may successfully 
achieve the same objectives without using a computer; 
computer-related projects should learn from these non- 
computer techniques and devices, as well as anticipate the 
possibility of being replaced by them. 

Tools for information processing in education and training 
(“a” above) should be constructed in anticipation of the 
use of such tools by the trainee or student later on, and 
the study of information processing as a direct component 
of the process of learning and teaching. 

2) Demonstration programmes have been talked about by 
NCET, but it has not been clear to me for whom the 
demonstration is to be made. The intended audience should 
be a considerable factor in determining the programme 
characteristics. In the end is it the potential sponsors, users, 
or the supporting citizen who is to be reached? What attitude 
and information should the intended audience acquire? Is it 
too soon for any kind of demonstration? 

3) Diffusion and the actual process of bringing about change 
in instructional process effectively to incorporate computers 
and similar technological devices has been implicit in con- 
siderations of demonstration. Are some uses sufficiently 
tested to merit introducing them in some 'big way into 
practice throughout the country? Should computer uses be 
developed in a way which anticipates and tries to work 
around possible obstructions to adoption by local 
education authorities? 

Of these the second two aspects are the most difficult 
problems faced in programme planning: who are the demon- 
strations for and how do we diffuse the results? Perhaps one 
of the more powerful ways to get things into the schools in 
this country is to get a demonstration working somewhere 
and let people see it. The schools have a rather complex 
structure and it is not possible to identify one person in 
each system throughout the country who must be reached; 
whether it is the director of instruction or the headmaster 
or someone else depends very much on the conditions and 
interactions in the particular area. Demonstration is an 
essential step in diffusion.’ 

Miss P.M. Ash said that education has changed over the last 
twenty to thirty years largely because we have come to 
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recognize the child as an individual. Production engineers 
are probably changing their attitudes because of the 
influence of education. 

Innovation in education is often two-way: first, it can begin 
in the classroom with an imaginative and creative teacher, 
and then spread to the rest of the school and outwards to 
other teachers; and second, ideas (after originating in the 
classroom) may be taken further, developed and refined by 
various agencies. These may be teachers’ centres, lecturers 
in Colleges of Education or Universities, curriculum develop- 
ment projects, study groups, in-service courses, or other 
agencies. 

I have great confidence in the teacher and I believe, given 
the opportunity, teachers will find new areas of exploration 
and exploitation for the computer. 

Teachers and lecturers should be involved in any project 
from the beginning, for it is they who, perhaps more than 
many of us, are in daily contact with children and young 
people learning. In addition, local education authorities 
should be involved for it will be they who will have to take 
on responsibility for implementation. If teachers, lecturers, 
and representatives of local education authorities are not 
involved from the beginning, projects to use computers in 
education will founder. 

Mr. Karl Zinn returned to his main theme: I would like to 
put the remainder of my framework before you as a guide 
to consolidating specific recommendations for managing 
the programme. 

Board of Management. What role does it play? What kinds 
of persons (and positions) best serve? What would be a 
desirable relationship with the Directorate; what is likely? 

Directorate. Success of a national programme (contrasted 
with other means to encourage information exchange 
among independent projects) will follow from the selection 
of good people for the Directorate, the assignment of 
authority, and then prompt action. 

Projects. Careful selection of projects must be followed by 
effective monitoring. How many projects can be supported 
by the monies available? Can evaluation be conducted in 
terms of the achievement of specific objectives and the 
checking off of milestones marking a desirable route to 
those objectives? What is to be the extent of ‘communica- 
tion’ among the projects: exchange of information and 
ideas; external testing of procedures and materials; 
sharing of common facilities? What are to be the priorities 
and management practices within each project? 

Public Relations. Eventual provisions for realization of 
beneficial changes in the schools should be anticipated 
from the start in an effective relation with the press and 
general public. 

Mr. A.M Hodgson commented that a view had been put 
forward that the management of the proposed two million 
pound scheme is simple and straightforward as long as there 
is clear accountability to individuals and not to committees. 



1 would like to suggest that things may not be quite so 
simple. Any mission-oriented programme must be clearly 
guided by objectives. An objective requires some clear value 
system and some clear criteria for assessing accomplishment 
of objectives. I submit that in such a complex educational 
mission, agreement on both these scores is vastly more 
complex than in an industrial mission. 

Secondly, it seems to me that individual accountability is 
difficult to realize in practice in this type of work, partly 
because of the extreme complexity of the problems and 
partly because of the interdisciplinary nature of their 
solutions. 

Mr. E.N. Adams: The project management plan may be 
weak in spite of thoughtful considerations at the higher 
conceptual level of overall programme planning. Three 
aspects of each project, whatever its goals may be, need to 
be weighed: software development; system operation; and 
application. 

In experimental projects, system operation tends to be at the 
mercy of the (system) software development people. One 
thing which must be decided at the beginning is who the 
machine is being run for. The system scientists will always 
be working on something that is new and interesting and 
will be ready next year; and the applications people will 
never produce without a machine that is operating today and 
nearly every day. 

Another likely conflict is between development of curriculum 
materials and development of instruction methods. The argu- 
ment of method development people is to wait on curriculum 
development until the methods are demonstrated to be 
effective and economical; on tjie other hand, without a first 
draft to use there is no place to begin the iterative process 
of developing effective and economical methods. 

In practice it is very hard to make a computer system work 
well. And when a shared system doesn’t work it is especially 
obvious. The manager of an operating system must have 
complete authority ; it can not be run by a directorate. The 
question is who has power over what is at the heart of these 
problems of pursuing research, development and service with 
the same system. I suggest appointing a single good manager 
for the operating system and a directorate responsible for 
the applications work done with it. All those in managerial 
positions might participate in negotiations about the 
properties of the system, but the actual running of the 
system can not be done by a committee. 

I doubt that the level of resources available will permit 
effective maintenance and use of more than one system, 
even if all the systems development work had been 
completed today. 

Mr. R. Seidel: One should bring in as team members, from 
the beginning, those people who will be responsible for 
implementing the materials being developed. They need 
training or, if you prefer, a reorientation toward the kind 
of change intended. And they should participate from the 
beginning of the sequence of research, development, testing 
and evaluation. Although there may not be much of a role 



for them in the beginning, toward the end, in the testing 
and evaluation phases, their support becomes essential to 
the success of the project. 

Mr. P. McLaren: The existence of a national programme 
should help motivate significant basic research, and the 
Directorate should review proposals and encourage funding 
from other sources for worthy projects. Experience with 
the applied programme will help isolate basic research 
projects. 

Anyone wishing to do work has at least two avenues to 
obtaining support: the n? Tonal programme, if funded, and 
the usual research councils. 

Mr. K. Zinn (after the Symposium): Much time was spent 
discussing the hardware requirements for various activities. 
Could four or five projects use the same facility, especially 
considering communications costs and unreliability? Is a 
larger system required for a research project, or for a 
‘systems’ approach to instruction which requires large data 



files for student records to be maintained on line? What is 
the value of maintaining separate efforts to explore alter na- 
tive system concepts? How long would it take to get a new 
system running? Will the manufacturer , on new systems, 
have solved many of the problems which an instruc*'onal 
system designer now struggles with on old equipment? 

To what extent can CAI uses be shared with others, e.g., 
computer literacy for persons outside science and 
engineering? Can data processing uses be shared with CAI, 
by time sharing, or by scheduling in long periods the sharing 
of a particular hardware system. 

Toward the end of the session McLaren reminded the 
participants of the important difference between what one 
would like and what one is willing to accept to work with. 
The hardware questions being discussed really are questions 
for the Directorate, to be decided during the first 6 to 18 
months using the advice of many persons, including those 
present at the Seminar. 
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Introduction 

Listening to the discussions this week my understanding 
of the directions in which developments of computer based 
learning should take place has alternated between periods of 
blinding lucidity and black despair — it is like trying to 
catch the soap in the bath, now you have it, now you 
don’t. 

It seems to me there are four broad goals to which research 
effort might be applied: 

to use CBL as a vehicle for psychological 
investigations into the learning process, 

to explore CBL to help solve real-life 
educational problems, 

to exploit the technical gadgetry to the full, 

to derive economic benefits in teaching. 

I would like us to explore this morning the balance of 
effort that should be applied to each of these activities. 

We have considered the subject from a number of points of 
view. I have been gratified at the way the papers and the 
discussions have interlocked, vindicating my rather 
arbitrary choice of session titles. John Annett will in a 
moment be recalling some of the questions we attempted to 
answer and some of the significant pointers we threw up. I 
would like us to examine the interdependencies of these 



aspects and consider what should be our scheme of priorities. 
My personal view is that we all too often think oniy in 
terms of CAI with the computer being a sort of buffer 
between the student and the teacher rather than the computer 
being a component in a tripartite student/ teacher/machine 
system. 

In deciding on the best course of aciion I would like to 
gauge your reaction to the idea of a concerted programme. I 
do not intend to invite you to draft a detailed set of 
projects, but would like your views in principle to a plan 
on the lines put forward by the National Council. I would 
like some indication of willingness to collaborate in such 
a scheme and under what safeguards and conditions. I would 
like further to explore opportunities for collaborative 
action on an international scale - the example of the joint 
project of Professor Le Corre and Professor Jones is 
particularly interesting in this respect. OECD is already an 
active agency in this fieid. Possibilities inciude: 

secondment of staff to projects overseas 

sabbatical tours to projects here 

international information networks 

international forums for discussion 

1 think we are bound to apply as much system to our own 
affairs as that we are advocating should be applied to the 
xhoolroom and 1 commend Dr. Seidel’s 7 point approach. 
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Statement 

One acknowledged purpose of this meeting to set out the case 
for a substantial investment in CAI research. If we are to 
convince the Government, and ultimately the public at large, 
that this activity is worthwhile, we must' first ask ourselves 
why we, as individuals and as a group, are convinced. Short 
term (e.g. 5 year) economic justification of CAI does not 
seem possible and there are many ways in which the sum we 
contemplate spending on research could be immediately 
applied to the benefit of education. If w? are completely 
honest with ourselves, most of us believe in the potential of 
CAI because education has to do with the transmission, 
collection, storage and retrieval of information and that 
computers constitute the foremost and the most powerful tools 
serving information processing. Whether we look ahead 5, 10 
or 50 years, the involvement of computers in education must 
be seen as the inevitable marrying of the tools to the needs. 

For this reason, I am not too disturbed by the possibility 
that support for research may not be immediately forth- 
coming. In the long term CAI, perhaps in some forms as yet 
not envisaged, will inevitably come about. 

This week we have been examining various aspects of CAI 
research as they may affect the outline plan for a 
coordinated research effort in the U.K. I shall try to 
summarize what have seemed to me some of the major points. 

Hardware and Engineering Problems 
During the period in which the research programme is likely 
to be operative, we expect rapid technical developments in 
the speed, capacity and cost of control processing units. 

On this count questions ot cost and efficiency are likely 
to look very different in five years time. We have something 
of a dilemma in the choice between starting cheaply with a 
class of redundant obsolescent machines such as the KDF9 or 
taking the best advantage of the newest CPUs. Possibly a 
compromise will be effected by using an ‘available’ machine 
in the very first instance but without tying the whole 
project to obsolescent hardware. 

It has been suggested that CAI may provide some inputs to 
CPU development. The older machines were designed with 
payroll and general ‘number crunching’ in mind and the 
possibility of more suitable CPUs should be explored. 

However, this aspect of hardware is not a serious obstacle* 
and much can be done with existing machines. More 
important are the student machine interfaces, the terminals. 

No one is satisfied with existing teletypes. CRT/light pen 
systems are still relatively expensive although the plasma 
tube offer some hope in the field of visual displays. Certainly 
research is needed in the design of more appropriate terminal 
hardware and a plea that the ergonomic requirements of 
terminals should be studied was met with general approval. 



Software Problems 

By and large appropriate computer software is available or 
can be developed for CAI use. This is not to say that what 
has been done could not be done much better and that new 
educational requirements will not bring new problems. One 
serious question is in the interchangeability of languages 
between systems. Discussions on common, or at least 
translatable, languages must be pursued. 

The “educational software” problems are not entirely 
distant from general control software problems. The lowest 
common denominator of CAI was referred to, rather scornfully, 
as “stuffing a textbook into a computer.” It is clear that 
putting a textbook into a computer is no solution to the 
educational software problem. There is a general recognition 
that CAI implies some kind of model of the learning process 
on which effective teaching strategies can be based. There 
has been some discussion as to whether adequate psychological 
models of learning exist. 

Currently experimental psychologists working in this area 
distinguish between theories of learning and theories of 
instruction. Skinner’s “learning” theory is strictly a theory 
of instruction in the sense that It is prescriptive, it tells 
you what kinds of teaching operations are necessary to 
achieve specified changes in behaviour. The aim is to 
refine and. adapt models of the instructional process and 
this can and should be done in the context of CAI. 

The days of competing theories, a decade or two ago, when 
“the learning process” was said to be of this kind or 
that, have passed. It now seems likely that ther° are various 
types of learning process. The question thus arises, can 
any type of learning be identified with a given subject 
matter? A number of ill-defined categories are in common 
use. We talk, loosely, of different subject matters (e.g. 
physics), different objectives ( e -g- knowledge), analysis and 
different “mental” processes (e.g. problem solving and 
skills). These cut across semantic boundaries and lead to 
confusion. A given subject matter (say, physics) can 
include a whole range of educational objectives and the 
psychological processes involved could be of many different, 
types. It does not, therefore, make too much sense to ask 
whether physics is suitable for a certain kind of instruction 
(say CAI). The educational and psychological characteristics 
of a given subject matter will constitute a unique mix of 
objectives and processes and will have this feature in 
common with other subject matters. It is implied that a 
variety of teaching strategies are likely to be involved in 
any given subject aiea. 

The group seems almost unanimous in the view that learning 
psychology is essential not just in preparing the way but in 






implementing CAI and that no amount of computing expertise 
will compensate for psychological naivete. 

The NCET Plan 

I have some anxieties about the implication of the NCET’s 
plan as set out in the feasibility study report. 



First, a bad precedent was set by the working party in not 
combining their analytical skills with suitable expert 
psychological knowledge. It is to be hoped that any R & D 
programme will not repeat the same weakness. Educational 
and psychological expertise are of the essence in getting 
the most out of CAI. 



Second, the proposed central type of organization for the 
project could lead to unimaginative and scientifically 
unproductive work. The project directors’ concern with 
the general selling objective of the project may lead to the 
temptation to produce short answers to pressing practical 
problems. The burden of justifying CAI in the short term 
could be counter productive in terms of worthwhile 
scientific objectives. 



Third, I am worried by the “takeover” attitude which 
colours the NCET report. I cannot accept the bland 
assurance that the existence of a £2m central project will 
not inhibit “free lance” research. It is quite possible 
that research councils would be inclined to discourage small 
projects on the general grounds that a great deal of money 
was already being spent in this area. In this conference, 
we have seen both the stultifying effect of large projects 
and the ingenuity of some independent Tesearchers working 
on small budgets. We could get the worst of both worlds in 
trying to attain the best. 



Finally, the NCET does not define CAI or refer t r me 
necessary relation between computer work and other efforts 
in educational technology, the psychology of learning, 
curriculum development and so on. There is a danger of 
intellectual isolationism in the structure of the project 
which must be avoided at all costs. 



In the ensuing discussion, Mr. Flood Page agreed that 
psychological research was important but pointed out that 
various research plans might work equally well. He 
suggested that, as in dentistry a “point of entry” was 
needed and that one point of entry was the automatic 
scoring of examinations by computer. He agreed with other 
speakers that we should soon have to think seriously about 
standardisation of computer languages. 



Professor Pask elaborated on the role of CAI in social develop- 
ment. Society is apt to run into catastrophes and the only way 
to avoid these is to pay attention to the thinking 
processes of man. CAI should give us not only quantity but 
quality in education and social awareness. CAI is not a 
trivial improvement in method, like a brighter slide 
projector. A non-trivial system should interact with the 
student to discover his learning plan and should measure 
the student’s relevant abilities such as short term 
memory. The system must then interpolate itself between the 



student’s learning plan and his measured competence. Ways 
of doing this must be specified before a big CAI system can 
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be built. Some of the present CAI systems are not adequate 
in the social context. For these reasons an essential part 
of CAI research should consist of experimenting with 
miniature systems to discover the basic design data for 
large CAI systems. 

Dr. Sleeman felt that more work should be done on adaptive 
teaching systems and in the general area of artificial 
intelligence. The Leeds project is pursuing this line in some 
of its work, for example. The Medical Diagnosis System. 
Pilot work of this kind should be done in preference to 
stuffing machines with instructional materials. If this was 
all large scale CAI meant, we would get nowhere in 3 or 5 
years. 



Mr. Hill liked the miniature system idea but suggested 
that one does not need much more than a pencil and paper to 
do it. The reasons for large scale demonstrations should 
not be confused with scientific requirements. There is a 
need for a pedestrian demonstration of what CAI is all about. 
What is needed is a programme which concentrates on a small 
number of systems and topics but which give rein to the 
intellectual capabilities of CAI. 

Mr. D’Arcy inteijected that much would depend on the 
calibre of the project director. The NCET plan requires a 
man of high calibre. 

Mr. McLaren held that the pragmatic approach must not 
be overstressed. He had in mind a parallel system in which 
CAI development and learning research ran in parallel and 
psychology would be important in this context. However 
multiple projects are wasteful and one-must bear economic 
realities in mind. 

Mr. Hodgson thought the directorate is going to have 
an impossible job if all the various aims are to be 
reconciled within the budget. Educational research is not 
the same thing as educational reform. Do we want analytic 
research or innovative research? Educational reform requires 
a large scale demonstration and entrepreneurial innovation 
requires financial backing. 



Professor Le Corre underlined the great deal of time . 
required to develop course material. The cost and difficulty 
should not be underestimated. Co-operat.on between 
countries, like that between France and Belgium in 
developing the physics course might help to economise in 
course development. It is also possible to begin with a 
good recognised text book. 

Mr. D’Arcy referred to the supposed failure of the 
educational system as a failure of the social system. CAI 
will not solve social problems. Care would have to be 
taken in introducing CAI so that it does not flounder. 

Dividing up the cake is going to reduce the effect of CAL 
There is no reason to suppose we can do better than the 
Americans, but one way of doing better lies in the effective 
organization and funding of projects. We have to choose 
educational rather than social problems, which can be 
solved, problems such as ancillary mathematics at University 
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level. In the schools CAI could be used in ‘modern’ maths 
which is an attempt to teach genuine problem solving. Good 
materials in this area are already being produced. 

Dr. Eraut spoke of the need to integrate CAI and 
other educational technology. A lot of people could work on 
various educational “subroutines” which could slot into 
non computerised courses. 

Professor Cook dissented from the view that the 
programme should concentrate solely on University maths 
when the same strategies might be used in sciences and 
medicine. He also underlined the need for serious work on 
student terminals. 

Mr. Hill thought it right to concentrate on secondary 
or University subjects and Mr. Duke said we should 
build on what has already been developed. Mr. Hill 
thought we could do better by doing co-ordinated research. 
Dr. Bate felt that the training of computer specialists 
had a lot to recommend it, especially as the topic was 
likely to appeal to Government and the computer industry. 
Professor Cook whilst agreeing that the U.S. had a 
great deal of experience, pointed out that in Europe and the 
U.K. we might have had more time to think and the project 
should take some account of this thinking. 

Mr. Duerdin explained that ICETT had not felt able 
to support the NCET proposal as a package. The “programme 
for action” should be quietly dropped but the proposals in 
chapter IX of the feasibility study should be pursued 
provided the whole programme is not based on the use of 
KDF 9’s. The Culham case might be an exception to get 
something moving rapidly. Industry can help a great deal as 
a user of the “products” of training and as a competitor 
for the resources of education. Industrial experience in 
forming and mounting viable long term projects could be 
valuable. Industry should co-operate in hardware design and 
work on student terminals is urgent if the price is to 
be brought down so that the use of CAI may be widespread. 

A kind of market research is needed in putting the 
“product” (CAI) to the needs of the market and industry 
has a great need in industrial retraining. 

Miss Ash suggested we could learn from American 
experience in using CAI for parts of courses or whole courses. 
It is preferable to consider whole courses but to use CAI 
only in those parts for which it is best suited. 



Mr. Crippin referred to developments in Scotland where 
many teachers were being trained in computer work, although 
not CAI as such. By familiarisation teachers may be taken 
along with the protagonists of CAI. Teachers are very 
active and much of the work in curriculum development 
comes direct from them. Whatever the short term potential 
of CAI, we cannot, in teaching, afford to ignore computing 
systems which involve processes similar to human thought 
but are in some ways much more powerful. 

Mr. Duke regretfully had to break into the discussion at 
this point. “It remains for me now but to bring this Seminar 
to a close. To me in many ways it has shown many of the 
characteristics of an ideal learning situation. The environment 
has been pleasant, relaxed and rich in experiences, and I am 
not only referring to the plumbing, the logistics have been 
unobtrusive and fortifying, the curriculum material has been 
fascinating. We have experienced a wide variety of teaching 
techniques from brilliant didactic exposition to group 
heuristics, opportunities for satisfaction of individual needs 
and curiosities have abounded, motivation has remained 
remarkably high and transfer great. 

This Seminar has also exhibited many of the characteristics 
of a system. The NEED for exchanging ideas was evident; 
the OBJECTIVES to outline the main problems and priori- 
ties. The CONSTRAINTS have been minimal and I must 
congratulate both those who have let their hair down, and 
to those who have managed to keep their’s on. Many 
ALTERNATIVE suggestions have been explored; the 
SELECTION of bons mots is now in the capable hands 
of our rapporteurs. In not too long a space of time I hope 
we will IMPLEMENT a report of these proceedings. The 
EVALUATION of the success of this Seminar is I feel already 
favourable and I sense many valuable feedback paths and 
communications nets have been set up to keep us all in 
touch with each others thinking. 

For my own part rny own horizons on computer based 
learning have been considerably broadened (and if nothing 
else I have learnt a lot about organizing conferences). Thank 
you ail for being such excellent participants; thanks, also, 
to Leeds University, our hosts; to the Leeds CAI Unit for 
putting on their demonstrations; to NCET and the US Office 
of Naval Research for their sponsorship; and to Dr. Jeremy 
Bray, Joint Parliamentary Secretary, Ministry of Technology, 
for his kind and encouraging remarks at our Conference 
dinner. I hope we all meet again before long.” 
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